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PREFACE 


In the fall of 2009, the National Center on Education and the 
Economy initiated a series of research programs designed to support 
our high school reform program, Excellence for All, based on our 
more than 20 years of research on the school reform programs of the 
countries with the most successful education programs worldwide. 


‘The design of Excellence for All entails the use of 
some of the world’s best instructional programs, 
with high quality curriculum and high quality 
matching examinations. To make this program work 
as designed, we had to be sure that the performance 
standards we identified as “passing” on the lower 
division exams we had selected in English and 
mathematics were sufficiently challenging that 
students passing these examinations were likely to 

be successful in the first year of a typical community 
college program. 


We asked the association of community colleges 
what that standard might be. They did not know. 
‘There was no shortage of opinion about what it 
might be, much of it based on asking panels of 
community college faculty for the answer. But 

this method of determining education standards is 
notoriously faulty, because educators, job foremen 
and others presumably in a position to know 
typically answer in terms of what they would like 
students and workers to know and be able to do, not 
in terms of what the program of study or the work 
actually requires. We quickly discovered that no one 
had done in-depth research on what was needed to 
be successful in our community colleges. 
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So we set in motion two empirical studies, one 
focused on English and the other on mathematics 
requirements. Each of these studies was guided by a 
panel of leading experts in that subject matter area, 
including key figures from the community colleges 
themselves, as well as leading subject matter experts 
and researchers. Both studies were overseen by the 
Excellence for All Technical Advisory Committee, 
whose members include many of the nation’s leading 
psychometricians, cognitive scientists, curriculum 
experts and testing experts. I am deeply indebted 

to both the subject matter committees and the 
Technical Advisory Committee for the time and 
careful attention they have given to these studies 
over the two-and-a-half years it has taken to conduct 
them. Special appreciation goes to the English 

Panel co-chairs, Richard P. Duran, Sally Hampton 
and Catherine E. Snow, for their leadership, 
thoughtfulness and creativity in steering the Panel 
through the challenging tasks we set before them. 


Most of the work, as is usually the case, was done 
by the staff. Betsy Brown Ruzzi, NCEE’s Vice- 
President for Programs, produced the original 
research design and has continued to be deeply 
involved in the work. Jackie Kraemer, Senior Policy 
Analyst, conducted the research. Jennifer Craw, 


Production Designer, assembled and aggregated all 
the data coding and developed the data displays. 
David R. Mandel, Director of Research and Policy 
Analysis, oversaw the whole process and played a 
key role in drafting the reports. 


This entire effort also enjoyed the support and 
encouragement of the Bill & Melinda Gates 
Foundation as part of their College Ready 
Education strategy. 


The nation is, at long last, engaged in serious 
discussion of what it means to be College and 
Career Ready. We believe that this research 
program will make an important contribution to 
that debate by cutting through strongly expressed 
opinions on the matter that turn out to be just 
plain wrong in the light of our findings, findings 
that may surprise many observers. We find, for 
example, that although challenging texts are used in 
community colleges, the reading tasks that are set 
before students are not especially demanding. Even 
so, many students attending community colleges fall 
short of the rather minimal literacy demands made 
upon them, casting an even harsher light on the 


failings of U.S. high schools. 


But these findings will not surprise all. As the facts 
presented in these reports came to light in the course 
of our research, I shared them with people very close 
to the institutions we were researching. Few of them 
were surprised. Most told me that the emerging 
picture corresponded closely to what they saw every 
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day in the field. They had long ago concluded that 
the debate about standards was unhinged from the 
realities in our community colleges. 


We offer these research reports in the hope that our 
findings will make an important contribution to the 
larger debate about what it means to be college and 
career ready and what our schools should be doing 
to provide curricula and instruction that will help all 
students be ready for college and careers when they 
graduate from high school. 


Some may charge that our findings constitute an 
argument to lower high school leaving standards. 
That would be a gross misreading of our findings. 

A large fraction of high school leavers cannot now 
do the work required of them in the first year of 

the typical community college program. Our first 
priority should be to reduce that fraction greatly by 
teaching all high school students what they will need 
to succeed in college. But we do not teach what they 
need, while demanding of them what they don’t 
need; furthermore, the literacy skills that we do 
teach and that they do need, we teach ineffectively. 
Many high school graduates cannot meet the 
typically limited literacy demands of community 
college programs. Perhaps that is the place to begin 
our deliberations. 


Marc Tucker, President 
National Center on Education and the Economy 
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lL SUMMARY 


THERE IS A STRONG CONSENSUS that 
students ought to leave high school ready for both 
college and careers. And there is little disagreement 
that being college and career ready ought to include 
being able to communicate clearly—to speak 
grammatically, write well and read the required 
materials with understanding. 


A large and growing fraction of our high school 
graduates go to community college either to launch 
their careers or to prepare for transfer to a four-year 
college. If a student leaves high school unable to 
succeed in the initial credit-bearing courses of the 
local community college, that student is ready neither 
for work nor college. And the fact is that a large 
proportion of our high school graduates are indeed 
unable to succeed in their first year in community 
college, and those who are assigned to remedial courses 
have a painfully low rate of college completion. 


So this report addresses a simple question: what 
kind and level of literacy is required of a high school 
graduate if that student is going to have a good 
chance of succeeding in the first year of a typical 
community college program? 


One would think that the answer to that question 
would be well known, but it is not. Community 
college staff have been asked for their opinions on 
that point, but people who study the process of 
setting standards of this sort know that people who 
are asked such questions typically answer based on 
what they would like students to know and be able 
to do, rather than on what the actual work demands. 


We present here an empirical analysis of the 

literacy skills needed in a range of initial required 
community college English and introductory courses 
in a diverse range of programs of study.? 


We began this research by randomly selecting a 
community college from each of seven states and 
then focusing on eight of the most popular and 
diverse programs in those colleges — Accounting, 
Automotive Technology, Biotech/Electrical 
Technology, Business, Criminal Justice, Early 
Childhood Education, Information Technology/ 
Computer Programming? and Nursing — plus the 
General Track.? We collected materials, including 
graded student assignments, tests and examinations 
from each college to allow us to analyze the reading 
and writing skills that are required in the initial 
credit-bearing courses in those programs, and in the 
first year English Composition course required by 
each program. We also analyzed the reading levels 
needed to understand the material in the textbooks 
used in those courses. 


1 A parallel study has been completed by NCEE analyzing the math- 
ematics needed for success. 


2 While there are distinct differences in the curricula of the Informa- 
tion Technology and Computer Programming courses we encoun- 
tered, the character of the texts they employ are quite similar so 
they have been joined together for analytical purposes. 


3 About one-third of community college students who graduate 
choose to major in the liberal arts and sciences, general studies 
and/or humanities, a figure that has remained steady over the last 
decade. The next most popular majors are in the health profes- 
sions and related clinical sciences, which encompass about 21% 
of all associate degrees granted. Business is another popular 
major, drawing 15.7% of community college students, followed 
by engineering at 6.5%. Security and protective services and 
computer and information services round out the most popular 
majors with 4.4% and 3.8% of students choosing these fields, 
respectively. While health fields have experienced an increase in 
graduates between the 1999-00 and 2009-10 school years (from 
15.3% to 20.9%), engineering has dropped from 10.5% of gradu- 
ates to just 6.5%. Most other fields have remained fairly stable. 
(NCES, Condition of Education, 2012, (2012). Washington, DC). 


THE ENGLISH LITERACY REQUIRED OF FIRST YEAR COMMUNITY COLLEGE STUDENTS 1 


We found that the reading and writing currently 
required of students in initial credit-bearing courses 
in community colleges is not very complex or 
cognitively demanding. While the information load 
of texts students encounter in community colleges 
is considerably more demanding than of those 
assigned in high school, students are not expected 
to make much use of those texts. The requirements 
for writing are marginal at best and the performance 
levels students are expected to meet with respect to 
reading are in many cases surprisingly modest. 


It turns out that the reading complexity of college 
texts used in initial courses is somewhere between 
the level of grade 11 and grade 12. One would think 
that this means that the level of the community 
college texts is comparable to the level of a student’s 
high school text and would therefore present no 
challenge to their reading ability. But that does 

not seem to be the case. Two things point in this 
direction. First, the high failure rates that students 
experience in community college suggest that these 
texts are too difficult for many of them to handle. 
Second, there are an accumulating number of studies 
of high school texts that point to their diminished 
level of challenge over the past half century at 

the same time as the demands of college texts are 
holding steady or increasing.* Taken together they 
suggest that high school students typically confront 
texts that fall short of those rated at grade 11 or 12. 


Our text complexity study noted that students who 
will be successful readers of information-rich texts 
written at the 11"or 12™ grade level must possess the 
following capacities: 


e The ability to read complex texts in 
unsupported environments; 


e ‘The capacity to process, retain and synthesize 
large amounts of new information; 


* ACT, ACT National Curriculum Survey (lowa City, [A: ACT, 2009). 
Common Core State Standards Initiative (2011). Common Core 
State Standards for English Language Arts & Literacy in History/ 
Social Studies, Science, and Technical Subjects, Appendix A: 
Research Supporting Key Elements of the Standards. Washington, 
DC: National Governors Association Center for Best Practice and 
Council of Chief State School Officers. 
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e Significant reading experience in a wide range 
of content areas; and 


e The ability to read and understand tables, 
charts, maps, lists and other documents that 
supplement the prose in many college texts. 


Many students emerge from high school without 
these capacities and experiences because reading 

for in-depth subject matter comprehension is not 
formally taught in our high schools and the reading 
that is required more often than not demands little 
more than searching for basic facts as opposed to 
trying to make sense of complex or conflicting ideas 
or both. The reading that is assigned in high schools 
is also drawn from much less complex texts than 
are found in community college, particularly in 
college courses focusing on technical areas such as 
information technology and automotive technology. 
Texts in these fields require the ability to read and 
understand technical vocabulary, charts and other 
visual representations of physical and mechanical 
phenomena not typically taught in high school 
outside of career and technical education courses. In 
many cases it is not that students might not come 
across such material, it is that they are rarely called 
on to engage with it. This disconnect between high 
school and college reading demands is particularly 
troubling and suggests a need to reexamine what 

is taught in high school. The Common Core State 
Standards in English Language Arts (CCSSE) 
address reading in history/social studies as well as 
science and technical subjects, and in so doing may 
increase the relevance of high school instruction. 


While the reading complexity of first-year 
community college texts is between 11 and 12" 
grade levels, we found that community college 
instructors typically make limited use of the texts 
they assign and use many aids (e.g., PowerPoint 
presentations, videos, outlines, flashcards) to help 
students understand the key points of the sections of 
the text they are asked to read. It would appear that 
students’ inability to read texts of the level assigned 
does not inhibit their success in their programs. Is 
this because the material in the texts is irrelevant 

to later success in education and careers, or because 


the instructors offer workarounds, recognizing their 
students’ limited reading ability?® 


The Programme for International Student 
Assessment (PISA) analytical framework used by the 
Panel to analyze the level of reading challenge makes 
a distinction between retrieval tasks — those that 
require a reader simply to find information and make 
basic interpretations of it — and analytic/synthetic 
tasks, that require the reader to reflect on and 
evaluate what they have read. Overall, we found that 
most of what first-year community college students 
are required to do falls in the former category. Only 
the English Composition classes reliably assign tasks 
that require students to reflect on and evaluate what 
they have read. 


‘The study also analyzed the reading and writing 
requirements found in tests and examinations in 
initial credit bearing community college courses. 
In this case, we found that most assessments in 
community colleges come in the form of multiple- 
choice questions that demand very little in the way 
of complex reading skills and no writing. 


Our analysis of the writing required to succeed 

in initial credit bearing courses in community 
college revealed that most introductory college 
classes demand very little writing; when writing 

is required, instructors tend to have very low 
expectations for grammatical accuracy, appropriate 
diction, clarity of expression, reasoning and the 
ability to present a logical argument or offer 
evidence in support of claims. 


To the extent that writing is required, it typically 
takes the form of informational writing or 
marshaling evidence for taking a particular course 


5 One explanation of what is occurring is that there has been an ele- 
ment of “pedagogical surrender” occurring on college campuses. 
Where once (mid-80’s to late 90’s) they taught to “the middle,” 
now they teach to “the base.” 


of action tied to a course-relevant profession. For 
example, filling in an auto repair order form, 
completing a pre-school class observation form, 
reporting engine analysis findings, writing up 
treatment notes for a nursing patient, or making 
an argument for taking a particular action on the 
basis of criminal justice system data. But this kind 
of writing gets the most modest attention in high 
schools, where literary analysis plays a significant 
role. However, even more worrying than how 

the balance is struck between different forms of 
writing in high school is that so little writing of 
any kind is assigned. Across various content area 
classes the default is short form assignments that 
require neither breadth nor depth of knowledge. 
Furthermore, the quality of instruction, especially 
with regard to argument, falls far short of what 
students need.® The good news here is that the 
CCSSE has recognized this problem and set out to 
address it by spelling out a much more ambitious 
approach to teaching writing, starting in the 
elementary grades and extending into secondary 
schools. But applauding new standards is not the 
same as enacting them. Serious attention at the state 
and local level to bridging the gap between where 
we are and where we need to be must follow and 
this should include greater attention to writing in 
teacher education across the board. 


With the exception of English Composition classes, 
complex writing plays a minor role in community 
college student exams. When writing is assigned 

in exams, the emphasis in grading is on the least 
cognitively demanding aspects of writing. At almost 
every turn one finds the weakness of high school 
writing being reinforced in community colleges 
when just the obverse ought to be the order of the 
day. Taken together this suggests that community 


& Applebee, A. N. and Langer, J. A. (2011). A Snapshot of Writing 
Instruction in Middle Schools and High Schools, English Journal, 
100.6” 14-27. 
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college could be a much more rewarding experience 
for students were it not for the weak preparation 
that precedes college and the modest expectations 
students encounter during their stay. 


We have noted that community college instructors 
do not expect their students to be able to read at 

the level of their texts or to write very much at all, 
suggesting that those instructors have very low 
expectations for their students, expectations so low as 
to deny many, if not most, students the opportunity 
to learn skills essential to the careers they have 
chosen to pursue. Conversely, we have also pointed 
out that nothing in the high school curriculum 
prepares students for some of what is expected in our 
community colleges. 


‘The response that many of our readers would no 
doubt expect from the Panel is a demand that 
community colleges raise their expectations for 
student reading and writing at least to the point that 
students be expected to read the texts they are given 
and to write material appropriate to the careers they 
have chosen at a level that goes beyond the simplest 
recall of facts to embrace the kinds of analysis 
expected of them on the job, and further, that the 
high schools be expected to prepare these students to 
meet such standards and to provide the foundation 
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skills required for their graduates to exercise the 
skills for which currently no foundation is provided 


in high school. 


Yes, but a note of caution is in order. We need to 
bear in mind that a very large fraction of high school 
graduates cannot meet the very low expectations that 
community colleges currently have of them. The 
nation may have to learn to walk before it runs, 
which means that it is important, first, to enable 

our high school students to meet the current very 
low standards before we ratchet those standards 

up. Nothing in that stance, however, should prevent 
the high schools from providing the skills needed to 
do the kinds of reading and writing now demanded 
by our community colleges for which no foundation 
is currently provided. Nor should it prevent 
community colleges from beginning now to assign 
more writing in those cases in which it now assigns 
none or from asking students to read material which 
is vital to their mastery of the initial skills their future 
employers will require. But, as they do this, they will 
also have to provide the support those students may 
need to succeed on these tasks. The aim here must 
be not to raise the standards come what may, but to 
increase student success on more demanding tasks 
that are vital to their success in their chosen fields. 


THIS REPORT SUMMARIZES one of 

several research initiatives the National Center on 
Education and the Economy (NCEE) is undertaking 
as part of its Excellence for All initiative. Excellence 
for All aims to bring what are known as aligned 
instructional systems to American high schools. 
Such aligned instructional systems are characterized 
by: 1) programs comprised of courses that constitute 
a coherent core curriculum, typically consisting, 

at a minimum, of courses in one’s native language, 
mathematics, the sciences, history and the arts, 

each of which is framed by a detailed syllabus; 2) 
instructional materials custom-tailored to support 
each curriculum; 3) teacher professional development 
that trains teachers to teach the curriculum and 
organize instruction so that the broad range of 
students likely to encounter it will succeed; 4) 

high quality examinations (typically dominated by 
essay and constructed response questions) that are 
designed to assess the extent to which the student 
has command of the material described in the 
syllabus and can apply it to unfamiliar problems; and 
5) professional scoring of the examinations. 


For more than 20 years NCEE has been 
benchmarking the national education systems that 
perform at the top of the international league tables 
and get large percentages of their students ready 

to be successful in college. A key finding from this 
research is that the nations that have overtaken the 
U.S. in student achievement have powerful, coherent 
and aligned instructional systems. This finding is 
no longer a secret, and in fact many researchers have 
converged on this conclusion. As NCEE has shared 
these results with states and localities it has found 
good numbers of them interested in putting such 
coherent, aligned systems into place, and in Fall 
2011 they began doing so. 
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BACKGROUND 


NCEE is now working in four states to pilot two 
different systems, ACT’s QualityCore program 

and the University of Cambridge’s International 
General Certificate of Secondary Education 
(IGCSE) program, in 39 high schools. One of the 
key provisions of the pilot is that students who 
succeed in these “lower division” (9 and 10" grade) 
programs and do well on their examinations will 
have the option to enter open admissions colleges 
without remediation. They could also stay in high 
school and complete a demanding “upper division” 
program that will prepare them for selective colleges 
and universities. 


Critical to this plan is to set trustworthy 
qualification scores on the English and mathematics 
examinations that certify that students are ready 

to succeed in initial credit-bearing courses at open 
admissions colleges and universities. This study was 
designed to support this effort by identifying what 
knowledge and skills in the English Language Arts 
are actually required in such courses. 


Within this broader effort, the specific goals for this 


project were: 


e To describe the literacy tasks in the initial 
required English course and the initial 
courses in a variety of program areas in open 
admissions colleges and universities; 


¢ To describe what the prerequisite literacy skills 
are for students to be successful in these courses 
given the demands of these courses; and 


e To inform the process of setting qualification 
scores on the aligned instructional systems 
examinations to enable students to succeed 
in credit-bearing courses at open admissions 
colleges and universities, given current demands. 


IN ORDER TO ADDRESS THESE ISSUES, 
NCEE collected course materials (syllabi, required 
texts, graded mid-term and final exams and, in some 
cases, graded assignments) from seven community 
colleges randomly selected from within each of seven 
states that are interested in this work. These colleges 
are in Arizona, Connecticut, Kentucky, Mississippi, 
New Hampshire, New Mexico and New York. They 
serve a mix of rural, urban and suburban populations 
and their enrollments range from 3,000 to 30,000. 


‘The standard way of determining what students 
need to know and be able to do to succeed in 
college is to put this question to highly regarded 
faculty. What they usually report, however, is 

their sense of what would be desirable rather than 
what is currently needed. To make sure we are 
reporting on what is actually required to do the 
work, rather than on the aspirations of community 
college faculty, NCEE gathered and analyzed actual 
evidence to determine the reading and writing 
knowledge and skills needed to succeed. This was 
done in eight highly popular and diverse program 
areas (Accounting, Automotive Technology, 
Biotech/Electrical Technology, Business, Criminal 
Justice, Early Childhood Education, Information 
Technology/Computer Programming and Nursing), 
as well as the initial mathematics and English 
courses required by each of these programs. We did 
not analyze any certificate programs, only programs 
that led to an AA, AAS, or AS degree, or that 
allowed students to transfer to a four-year institution 
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METHODOLOGY 


to continue studying for a BA or BS degree. In each 
case, the required courses for the general track were 
covered in the set of courses we analyzed. 


In order to analyze the literacy demands in the 
courses we selected (i.e., the first required English 
course as well as the introductory or 101 courses 
for each program), a panel of literacy experts drawn 
from community colleges as well as from four-year 
institutions and high schools was assembled (see 
the Biographical Sketches of the panel members in 
Appendix A). 


To review and analyze the evidence, three different 
studies were undertaken: 


e An analysis of the complexity of the texts 
used in the English Composition and initial 
introductory courses; 


e An analysis of the complexity of reading tasks 
based on texts used in these classes in order to 
understand what students are asked to do with 
respect to their assigned reading; and 


e An analysis of a sample of graded writing 
assignments collected from these classes 
in order to understand what kind of work 
generally receives grades of A, B and C. 


IV. FINDINGS 


Text Complexity 


THE FIRST READING STUDY was an analysis 
of the complexity of textbooks commonly in use 

in entry-level community college courses, using 
four readability measurement tools to analyze and 
compare representative texts.’ These measures 
included three of the leading quantitative tools 
(Flesch Kincaid, Dale-Chall and Gunning Fogg) 
as well as the Dale-Chall qualitative tool. In order 
to do the qualitative analysis, a small panel of high 
school and community college instructors was 
recruited to review a selection of samples from each 
of eighty-six textbooks that were collected from 

the seven colleges in this study. The Dale-Chall 
qualitative tool included rubrics for each of six text 
types (Literature, Popular Fiction, Life Sciences, 
Physical Sciences, Narrative Social Studies and 
Expository Social Studies). The Panel was faced 
with a decision about the Information Technology, 
Auto Mechanics and Mathematics texts that did 
not map neatly onto any of these text types. They 


decided that Auto Technology was similar enough to 


Life Sciences to take advantage of the Life Sciences 
rubric but decided that Information Technology and 
Mathematics should be left out of this qualitative 
analysis as none of the rubrics fit the characteristics 
of the texts used in these courses.® 


‘The study found that, across the disciplines, the 
textbooks present comparable and significant 
challenges for students who will be using them 
in largely unassisted environments (i.e., assigned 


7 Specifics on the selection and application of the measurement 
tools are found in the full Text Complexity Report, Appendix B. 


8 \t turned out that the absence of this data was likely of modest 
consequence as the findings across the three non-Gunning Fog 
measures tended to be quite close to one another within each 
program of study. 
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readings outside of class as opposed to supported 
readings in the classroom environment). The 
texts are laden with content and are otherwise 
characterized by extensive specialized vocabulary, 
sophisticated text structures and long, relatively 
complicated sentences. Texts in many of the 
disciplines studied require significant amounts 
of background and concept knowledge for 
comprehension, and complex concepts and 
information are often presented in tables, charts, 
maps and lists that supplement the prose. 


Chart 1 on the next page shows the average grade 
level of texts on each of the four tools in each of the 
program areas. The Gunning Fog tool, although 
correlated with the other tools, shows a higher 
complexity level consistently across all areas as it 
overweighs sentence and word length as the key 
measure of complexity and takes less account of 
vocabulary and word frequency levels. As a result 
the analysis team decided to set the Gunning Fog 
measures aside and focus on the others. Among 
the remaining three tools the overall average level 
was about grade 11.5, with Criminal Justice, 
Biotechnology and Nursing rated as high as grade 
13 and English Composition and Early Childhood 
Education as low as 9.5. The study noted that several 
of the fields analyzed are highly technical (e.g., 
Auto Technology and Accounting) and none of the 
tools available for text analysis take into account 
the specialized technical vocabulary and the visual 
representations that are typically found in these 
texts. Consequently, they surmised that these texts 
would have scored higher in complexity if these 
factors were taken into account and also observed 
that these types of texts are not found in most high 
school curricula. 


CHART 1 Text Difficulty Comparison: Grade Ratings for Texts in Introductory Courses in Select 


Programs of Study® 
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° English Composition texts were rated on two different grounds. First, they were based on the text complexity in the body of the textbook 
texts. Second, they were based on excerpts found in these texts (from literature, sample letters, etc.) employed to illustrate important ideas. 
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We concluded that students who will be successful 
readers of these texts must possess the 
following capacities: 


¢ The ability to read difficult texts in 
unsupported environments; 


e ‘The capacity to process, retain and synthesize 
large amounts of new information; 


e Significant reading experience in a wide range 
of content areas; and 


e The ability to read and understand tables, 
charts, maps, lists and other documents that 
supplement the prose in many college texts. 


Many students emerge from high school without 
these capacities and experiences. And it is no 
wonder as reading in subject matter areas is not 
formally taught in high schools, and the reading 
that is assigned is typically from much less complex 
texts that have seen their challenge levels decline 
for the past several decades at a time when the 
workplace is trending in just the opposite direction. 
Consequently, we are witnessing a difficult situation 
only getting worse when a sharp course correction 
is urgently needed. Today’s high school texts also 
suffer by rarely including the types of charts and 
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visual representations common in many college texts 
in technical fields. Taken together this disconnect 
between high school and college reading demands 

is stark and points to reexamining what is taught 

in high school. The CCSS could play a constructive 
role in this process as they address graphic literacy 

as part of their Standards for Literacy for History/ 
Social Studies, Science and Technical Subjects but not 
as part of their Standards for English Language Arts. 
This is probably as it should be as it is in the sciences, 
technology courses and in the social sciences and 
mathematics as well where there are naturally 
occurring authentic opportunities to provide 
students with more encounters with charts and 
visual representations of the natural and built worlds. 


It is notable that college instructors in our study 
described different ways that they addressed the lack 
of preparation among their students for reading. 
Their strategies include creating PowerPoints and 
outlines, treating the text as a review resource for 
foundational information covered in class, and even 
making the texts optional. While these approaches 
can be seen as understandable, even commendable 
efforts to promote student learning, the end result 
is that many of our college students are likely left 
without the skills to handle the reading demands 

in the classes they will take subsequently, and very 
likely in the workplace as well. 
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V. FINDINGS 


THE SECOND READING STUDY attempted 

to answer the question: what are students asked to 
do with what they read? To approach this question 
the panel looked at a set of reading tasks assigned in 
community college courses. Instructors were asked 
to supply a set of three tasks, if available, one from 
early in the semester, another from mid-semester, 
and one that they considered the most challenging of 
the semester. Panelists were provided with the sample 
of the background reading for each task (if available), 
a copy of the actual assignment, and samples of 
student work produced in response to the task (if 
available). They were also provided for background 
purposes with whatever other class materials had 
been supplied, including syllabi and transcripts of 
interviews with instructors. 


‘The panel rated the complexity of these tasks using 
an adaptation of the Programme for International 
Student Assessment (PISA) reading scales, which 
organize task complexity in three categories - 
Retrieve and Access, Integrate and Interpret, and 
Reflect and Evaluate. This system of categorization 
was selected because it is widely known and has 
widespread credibility. The PISA rubric, that 
distinguishes seven levels of complexity within 
each task category, was designed to classify items 
on a literacy assessment, and thus it crossed two 
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dimensions — text burden and task burden — 
into a single rubric. 


Given that the application here was somewhat 
different, a number of modifications to the PISA 
rubric were needed to make it usable for our 
purposes. First, the seven complexity levels were 
collapsed to five, to sharpen the distinctions between 
levels.!° In the course of the review it became clear 
that the demands of the texts that framed the 

tasks were not related in any simple way to the 
complexity of the tasks themselves, but this is a 
built-in characteristic of the PISA scales. This lead 
the panel to develop a context level progression scale 
(derived from the PISA reading scales), to reflect 
how demanding was the challenge of finding and 
understanding the information to be retrieved, 
integrated or evaluated. Adding this in as a second 
dimension independent of the challenge of the 
retrieval, integration or evaluation task allowed 

an unconstrained analysis of these two aspects of 
reading. We thus ended up with two rubrics to 
employ for this analysis (see scales on the next page). 


10 |t should also be noted that the PISA rubric, elegant as it is, was 
designed for test items, not tasks based on lengthy or multiple 
passages, so some adaptation was needed to use it for 
these purposes. 


READING 


TASK COMPLEXITY SCALE (ADAPTED FROM PISA) 


Access and Retrieve 


Integrate and Interpret 


Reflect and Evaluate 


5 Combine multiple Demonstrate a full and Hypothesize about or critically evaluate 
pieces of independent detailed understanding of the a text, taking into account multiple 
information in an whole text or specific sections. criteria or perspectives, and applying 
accurate and precise Make multiple inferences, sophisticated understandings from 
sequence or structure. comparisons and contrasts beyond the text —i.e., bringing a 

that are both detailed and situation model to bear on the textual 
precise even in the face of interpretation. In service of evaluation, 
unfamiliar abstract ideas refer to appropriate dimensions of text 
or competing information. (e.g., genre, presupposed expertise, 
Generate abstract categories appropriateness for specific audiences). 
for interpretations. 

4 Locate and possibly Demonstrate a full and Hypothesize about a text, drawing on 
combine several pieces detailed understanding of specialized knowledge (a situation 
of information, some of a text even in the face of model) and on deep understanding. 
which may be outside the ideas that are contrary to Critically analyze and evaluate 
main body of the text. expectations. Construe the potential or real inconsistencies, 

meaning of nuanced language. either within the text or between the 
Apply criteria to examples, text and ideas outside the text. 
using high-level inference. 

Generate categories to 

describe relationships between 

parts of a text. 

3 Locate several pieces Use text-based inferences Use formal or public knowledge (a 
of information, each of to construe the meaning of widely shared, nontechnical situation 
which may need to meet a section of a text by taking model) to hypothesize or critically 
multiple criteria. May need into account the text as a evaluate a text, thus showing accurate 
to combine information of whole, and/or to extract and understanding of the text. 
different kinds (e.g., verbal understand categories relevant 
and graphical). to the textual inferences. 

2 Locate several pieces Integrate several parts of Make connections or comparisons, 
of information, each a text in order to identify give explanations, or evaluate a 
of which may need to the main idea, understand feature of a text. Demonstrate a 
meet multiple criteria. a relationship, or construe detailed understanding of the text 
Combine pieces of the meaning of a word or by relating it to familiar, everyday 
information within a text. phrase. Compare, contrast knowledge (a simple situation model) 

or categorize taking several or a less detailed understanding that 
criteria into account. draws on less common knowledge (a 
more sophisticated situation model). 

1 Locate one or more Recognize the main idea or Make a simple connection or 
independent pieces author’s purpose in a text, comparison between information 
of explicitly stated making low-level inferences. in the text and common, everyday 
information meeting knowledge (personal knowledge 
a single criterion, by schemas), or explain a feature of 
making a literal or the text by drawing on personal 
synonymous match. May experiences or attitudes. 
make simple connections 
between adjacent pieces 
of information. 
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CONTEXT LEVEL PROGRESSION SCALE 


Level Description 


5 Target information has all the challenges spelled out in 4 and, in addition, comes from different parts 
of a text and/or from different sources within a text (e.g., tables, graphs, glossaries), and is likely to be 
unfamiliar to the reader. 


4 Target information is deeply embedded in the text, and competing information is strongly distracting. The 
text is long, complex and contains claims contrary to expectations, including potential or real inconsistencies. 


3 Target information is not prominent in the text that may include ambiguities, is complex or long, and 
competing information is extensive and prominent and/or distracting. 


2 Target information is relatively accessible, but the text includes competing information. 


1 Target information may not be prominent in the text, but is easily found, there is little or no competing 
information, nor is abstruse discipline-specific knowledge required. 


There were 54 tasks to be analyzed. Each was If panelist ratings were divergent, they discussed 
assigned to two panelists and panelists were asked their reasoning to see if they could agree on a 

to rate each task on each category of the PISA scale common rating. In cases where they did not agree, 
that fit the task. They assigned a 0 for any category a third rating provided by one of the three co-chairs 
they found that did not apply to a particular task. If became the rating of record. 

a reading sample was provided (or if the assignment 

asked for students to choose a text of a certain type Chart 2 shows the average complexity level for the 
that panelists were familiar with, such as song lyrics analyzed tasks by program area.'! The ratings average 


or an article from a specific magazine), they were 
asked to assign a context level to the text as well. 1 Ratings of zero were not included in the averages. 


CHART 2 Average Reading Task Complexity Ratings by Type and Program 
5 


i Access and 
Retrieve 


) Integrate and 
Interpret 


i Reflect and 
Evaluate 
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at about a level 2 of 5 save for English Composition tasks was consistent (see Chart 3, next page). The 


and Biotechnology where higher ratings were found. following examples of an introductory course task 
and an English Composition course task are typical. 

What this chart does not show (but what is clear Example 1 is from an IT class. It asks students 

from the program by program charts) is that many to access and retrieve information from multiple 

tasks have a Reflect and Evaluate rating of 0, with sources and perhaps integrate information at a 

the notable exception of English Composition tasks rudimentary level. But the nature of the assignment 

and the few tasks collected from a Biotechnology precludes deep reflection or evaluation. Example 2 

course. This divide between the English is from an English Composition course. It asks the 

Composition tasks and the introductory course students to write a “blog entry” that analyzes the 

EXAMPLE 1 


CiS111 LAB FALL 2010 PASSWORDS 


1. 


Think of a catchy phrase or word to encrypt into a password. Using the heuristics outlined in chapter 
12 convert your password into one that another user may not guess. Use at least three changes. 
Obviously, do not use one that is an active password. 


. You are a manager in a business department coordinating a password security plan for the Network 


Administrator. Consider, password format guidelines, expirations, entry lockouts, password disclosures 
and any other important details. Source your resources. Write a one-page summary detailing your plan. 


. What antivirus software is installed on your computer? How do you update the software to download 


the newest virus protection? Do a complete system check. Answer: How long did it take? How many 
files were checked? Were any viruses found? 


EXAMPLE 2 


BLOG #2: ANALYSIS PRACTICE 


Review the lyrics and corresponding cover art for The White Stripe’s “Blue Orchid,” Bruce Springsteen’s “Born in 
the U.S.A,” and Lady Gaga’s “Paparrazi.” Select ONE of the three and complete the following steps in writing: 


1. 


2. 


Analyze the written text and interpret the song’s meaning, referring directly to the lyrics. Do not summarize. 


Analyze the visual image and interpret the album cover’s meaning, referring directly to the artwork. 
Do not summarize. 


. Analyze the relationship between the text and the visuals. In what ways is the album cover a 


representation or misrepresentation of the song lyrics? How do they influence one another and 
your experience of them? What deeper understanding do you now have about the visual and text when 
experience together? 


Note: You do not have to have prior knowledge of any of the artists or songs provided. In fact, you should avoid using any research or prior knowledge about 
the music and instead see what conclusions you can draw by studying only the text and visual with your current life experience. Your blog response must be 
AT LEAST 250 words long. 
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CHART 3 Average Reading Task Complexity Ratings for English 
Composition Courses and Introductory Courses 


i Introductory Courses 


8 English Composition 
Courses 


Access and Retrieve 


Integrate and Interpret 


lyrics and cover art for an album and reflect on the 
relationship between them. This assignment clearly 
asks students to reflect on and evaluate information. 


English Composition tasks were not only notable 
for including far more reflect and evaluate tasks 

but overall had much higher scores on the task 
complexity scale than the introductory courses. 

It was suggested by panel members that English 
Composition courses, unlike introductory program 
courses, tended to focus on analysis of text rather 
than straightforward information retrieval. Some of 
the panelists noted that the assignments for many 
introductory classes were not especially detailed 


Reflect and Evaluate 


regarding expectations for students and left room 
for students to take a less analytic stance. Often 

the panelists said they were not clear what the 
instructors were expecting until they saw the student 
samples that instructors deemed acceptable. On the 
other hand, the English Composition courses often 
included detailed rubrics for each assignment. 


To provide another angle on the demands of the 
reading tasks that students encounter in their first 
year of community college, assuming they are not in 
developmental courses, we have created three charts 
(4A, B and C, below) that display how the ratings 
were distributed over the PISA inspired 1-5 scale for 


CHART 4A Distribution of Reading Tasks by Complexity Rating for Criminal Justice Courses 
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1.5-2 2.5-3 


3.5-4 4.5-5 


CHART 4B Distribution of Reading Tasks by Complexity Rating for Early Childhood 
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CHART 4C Distribution of Reading Tasks by Complexity Rating for English Composition Courses 
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English Composition courses and two other sectors, In addition to the divide between the English 
Criminal Justice and Early Childhood Education, Composition and introductory courses, we also 
to provide an illustration of what was found in the looked at the relationship between the context 


courses other than English Composition. 


level of the texts and the complexity of the tasks. 
Not surprisingly, we found that tasks were more 


Here again the sharp distinction between English complex as the demands posed by the texts they 
Composition and everything else is readily apparent are linked to, their context levels, increased. This 
as are the modest demands students encounter in is shown in Chart 5 (next page) where we see 


courses other than English Composition. 


evidence of a clear trend of context level rising 
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as reading task complexity rises for “Access and of the Technology and science-oriented classes. With 


Retrieve” and “Integrate and Interpret” tasks. The access and retrieve and integrate and interpret tasks 
same phenomenon was also found for “Reflect and found in 101 courses receiving an average rating at 
Evaluate” tasks for every interval but the first. or slightly above 2, it appears that students in these 
courses are hardly being challenged at all to learn 
Overall, this analysis suggests that college students what such courses could offer. This can’t serve them 
are rarely asked to do complex analyses of texts, well as they continue their studies, apply to transfer 
except in English Composition classes and in a few to a four-year college, or enter the workplace. 


CHART 5 Average Reading Task Complexity Rating by Context Level 
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VI. 


FINDINGS 


FOR THE ANALYSIS of writing demands in 
community college, the panel analyzed samples 

of student writing from English Composition 
classes and other classes that required writing. In 
this analysis of writing that is assigned to students 
versus other forms of writing they may undertake 
on their own, it is notable that there was very little 
writing required in several of the program areas 
such as Auto Technology, Nursing and Accounting. 
In several other areas, like Criminal Justice, 
Information Technology/Computer Programming 
and Business some classes at some colleges required 
papers or essays on exams but many classes required 
only multiple choice midterm and final exams with 
no written work. Overall, 14 (33 percent) of the 43 
introductory courses we examined had exclusively 
multiple choice or true/false exams and assignments. 
A good many more were primarily assessed using 
multiple-choice items, with the addition of a very 
limited set of short answer questions or a single 
essay aimed at no more than measuring a student’s 
ability to recall isolated concepts. !? 


We collected a set of varied writing samples. While 
the English Composition courses, from which most 
of the essays were collected, assign fairly traditional 
essays, many of the introductory courses assign 
specialized writing that is focused on a particular 
vocational area, such as a car repair form for an 
Auto Technology class, a case study form reporting 
a home visit to a preschool student in an Early 
Childhood Education class, or a business plan 
from a Business course.!? In addition, across most 
of the program areas we also encountered writing 


12 See Appendix C for a chart detailing evaluation and exam formats 
used by the 101 courses. 


13 See Appendix D for an excerpt of a business plan that received a 
high grade. 
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samples from timed final exams; but these were 
hard to compare to essays that students could revise 
and spend more time on. There was discussion of 
whether or not the “essay” rubrics could be used 
for this type of writing. It was decided the essay 
rubrics could be used but would be adapted as 
needed for different writing formats. For example, 
the panelists did not use the “introduction” category 
for a case study or technical form. There was also a 
question raised whether students’ handling of the 
content of the course carried significantly more 
weight than the quality of their writing to such an 
extent that it might distort the grading of writing 
from introductory program classes. This does not 
appear to be the case. The panelists, none of them 
specialists in the content area courses, likely valued 
writing quality more than content correctness 
across all the writing topics. Nonetheless, when 

we compared panel vs. instructor “grades” in these 
courses, the relationship was not different from that 
found for English Composition courses. 


‘The panelists analyzed the writing samples using 

a rubric they developed that drew on the CCSS in 
English Language Arts and the Cambridge IGCSE 
English First Language rubric that was seen by the 
Panel as being especially well crafted. The Panel 
developed three versions of the rubric representing 
the types of writing most commonly assigned in 
college: narrative, information and argument. All 
three rubrics use the same categories: Introduction; 
Organization; Development; Vocabulary; Closure; 
and Other. For timed essays and other specialized 
formats panelists did not include any category 
(like Introduction) that did not apply. Each rubric 
includes 4 score point levels (see rubrics below). 
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‘The three rubrics were used to serve multiple 
purposes. First, to understand and describe the kind 
and quality of writing that is being produced in 
community college entry-level courses. Second, to 
determine what level of writing is required to earn 
grades that signal success in college and suggest that 
a student has a good chance to complete a program 
that will yield a certificate or degree. This, in turn, 
promised to generate a strong signal about the 
literacy level students need to be college ready. 


To move forward with this college-ready analyses 
two decisions had to be taken: what score points on 
the writing rubrics could be equated with college- 
readiness; and what instructor grades signaled 
college success? The answer to the first question was 
judged to be 2.5 on each of the 4-point scales as 
performance at 2 was viewed as inadequate, 3 was 
viewed as successful, and there is the possibility that 
some performances south of 3 yet north of 2 might 
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lead to success in subsequent courses. On the second 
there was general agreement that college success 
would most likely require student performance at 

a GPA of 2.75 or above. This score was based, in 
part, on the NCEE Technical Advisory Committee’s 
(TAC) choice of B- as a qualification score for 

likely success in college. The TAC’s choice of this 
performance level was strongly influenced by the 
college readiness benchmark research conducted by 
the College Board and ACT correlating students’ 
scores on their college readiness exams with grades in 
college that would indicate a good likelihood (a 67% 
chance) of students achieving in their first semester 
at college an average GPA of B-.4 


14 Andrew Wiley, Jeffrey Wyatt and Wayne J. Camara, The Develop- 
ment of a Multidimensional College Readiness Index; (New York, 
NY: The College Board, 2010). Jeff Allen & Jim Sconing. Using 
ACT Assessment Scores to Set Benchmarks for College Readiness, 
(lowa City, IA: ACT, 2005). 


SCORE POINTS - ARGUMENT 


Introduction 


Score Point 4 


The introduction to the 
paper’s claims builds 
context for the argument 
by presenting data, 
employing narrative, or 
through the use of other 
rhetorical strategies. 


Score Point 3 


The paper at the outset 
engages and orients 
the reader by taking a 
stance or presenting a 
dilemma or conflict to 
be resolved. 


Score Point 2 


The paper contains an 
introductory statement 
or section that makes 

a claim(s). 


Score Point 1 


At the beginning there 
is an attempt to make 
a claim but meaning 
is hampered by lack of 
language control. 


Organization 


The organizing structure 
and transitions between 
the essay’s ideas and 
sections enhance the 
essay’s argument. (One 
idea builds upon the 
next; the flow of the 
argument is clear and 
logical.) 


A logical organizing 
structure links the 
claim(s), reasons and 
evidence. 

Transitions effectively 
convey the relationships 
among ideas and link 
the major sections of 
the text. 


Reasons and evidence 
are provided but may 
not be presented in an 
orderly manner that 
links the evidence to 
the claim(s). 
Appropriate transitions 
are used between 
paragraphs. 


The overall structure 
is inadequate with 
organizational 
problems obscuring 
the central idea. 


Development: 


Sound reasons and 


For the most part, 


There is variable 


The paper may provide 


¢ Claims sufficient relevant appropriate reasons development of the a simple list that mixes 

eR evidence support the and sufficient relevant | logic line. Minimal and | facts and reasons or 

easons : “ : : ; 
; claim. evidence support the sometimes irrelevant | just lists reasons. 

e Evidence | Each component is claim. material is presented. 
employed in support of | The paper contains no 
the essay’s argument. extraneous information. 

The essay may address 
counterarguments to its 
central claim. 

Vocabulary The paper contains clear, | The paper contains The paper contains The paper contains 
appropriate and precise | clear, appropriate and mostly everyday everyday language 
language. precise language and language, but also related to the topic. 
Issue specific language | some issue specific includes some precise 
is used with accuracy vocabulary. language related to 
and in a way that the development of 
supports the authority of the argument. 
the paper’s argument. 

Closure The concluding section A concluding A concluding The paper simply ends 
attempts to synthesize statement or section statement is without attention to a 
the essay’s ideas in a that supports the presented, although it | conclusion or closing 
way that reinforces its arguments or opinions | may not flow logically | thought. 
central arguments or found in the essay. from the argument or 
opinions. opinions in the essay. 

Other Punctuation and Punctuation and There may be frequent | The paper contains 


grammar are used 
competently and 
sometimes with a flair 
that enhances the voice 
of the essay. 

A range of complex 

and varied sentence 
structures enhance the 
paper and reflect the 
writer’s voice 


grammar are used 
competently. Although 
there may be some 
distracting errors, 
meaning is clear. 
Sentence structures 
are for the most part 
complex and varied. 
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and significant errors, 
but they do not inhibit 
understanding. 
Simple sentences are 
the dominant syntactic 
form, although there 
is some evidence 

that the writer 

has tried to write 
more sophisticated 
structures without 
SUCCESS. 


frequent and significant 
errors in spelling, 
punctuation and 
grammar, but some 
meaning is intelligible. 
The paper contains 

a minimal number of 
grammatically accurate 
sentences. 

Lack of language 
control is evident 
throughout. 


DENTS 


19 


20 


SCORE POINTS - INFORMATIONAL WRITING 


Score Point 4 


Score Point 3 


Score Point 2 


Score Point 1 


Introduction 


The paper at the outset 
engages and orients the 
reader by announcing the 
topic; strategies for setting 
a context for the essay are 
present and integrated with 
the material that follows. 


The paper at the outset 
engages and orients the 
reader by announcing the 
topic and the (perhaps 
general) context for what 
follows. 


The paper contains an 
introductory statement 
that announces the 
topic. 


The paper may 
provide a simple 
generalization about 
the topic or simply 
begin abruptly. 


Organization 


Complex ideas, concepts and 
information are organized 
into categories that reflect 

a solid engagement with the 
subject. 


Complex ideas, concepts 
and information are 
organized into categories. 
Transitions effectively 
convey the relationships 


Related ideas are 
grouped. 

Appropriate transitions 
are used between 
paragraphs. 


The ideas may 

be organized as a 
simple list. 

Ideas may be linked 
within sentences and 


punctuation and grammar 
is evident, with few if any 
errors. 

Sentence structures are 
complex and varied. 
Formatting and graphics 
(if included) enhance 
comprehension. 


punctuation and grammar 
is evident, although there 
may be some distracting 
errors. 

Sentence structures are 
for the most part complex 
and varied. 

Formatting and graphics 
may be provided to aid 
comprehension. 


punctuation and 
grammar is in evidence. 
Although there are 
frequent and distracting 
errors, they do not inhibit 
understanding. 

Simple sentences are the 
dominant syntactic form 
although there is some 
evidence that the writer 
has tried to write more 
sophisticated structures 
without success. 


Across the major sections of | among ideas and link paragraphs. 
the text, ideas are connected | the major sections of the 

in ways that enhance the text. 

reader’s understanding. 

Development | The topic is developed with | The topic is developed Uses facts, details, The paper is 
well-chosen, significant with significant and definitions, quotations | somewhat developed, 
and sufficient details, sufficient details, facts, | or other examples but may not provide 
facts, definitions or definitions or other to develop the topic the most significant 
other examples. There is examples, and offers through synthesis, or relevant facts 
appropriate (to the content) | explanation, elaboration | explanation, elaboration | or examples. 
and well articulated, or comparison across the | or comparison, but Expected syntheses, 
explanation, elaboration or essay. development is uneven. | explanation, 
comparative development elaboration or 
across the essay. comparison difficult 

to discern. 

Vocabulary The paper contains precise | The paper contains The paper contains The paper contains 
language and domain precise language some precise language | everyday language 
specific vocabulary. and domain specific related to the topic. related to the topic. 
The domain specific vocabulary. The domain 
language is used with specific language is used 
control. with uneven control. 

Closure The conclusion presents The conclusion moves The paper contains a The paper simply 
a synthesis by logically toward a synthesis by concluding statement ends without 
connecting the facts and logically connecting the or section. attention to a 
ideas addressed in the facts and ideas addressed finding, conclusion 
development of the essay. in the development of the or closing thought. 

essay, but the logic may be 
less than sound, and the 
attempted synthesis flawed. 
Other Competent use of Competent use of Knowledge of basic There are weaknesses 


in spelling, 
punctuation and 
grammar, but 
some meaning is 
intelligible. 

The paper contains 
a minimal number 
of grammatically 
accurate sentences. 
Lack of language 
control is evident 
throughout. 
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SCORE POINTS - NARRATIVE 


Introduction 


Score Point 4 


The paper engages and 
orients the reader by 
presenting a context, 
by introducing a 
narrator or character, 
or by entering 
immediately into the 
action strategically or 
dramatically. 

An engaging narrative 
voice is immediately 
present on the page. 


Score Point 3 


The paper engages and 
orients the reader by 
presenting a context, by 
introducing a narrator or 
character, or by entering 
immediately into the 
action strategically or 
dramatically. 

There is the presence of 
a narrative voice in the 
opening. 


Score Point 2 


The paper presents a 
context for what is to 
follow and introduces a 
character or narrator. 


Score Point 1 


The paper may 
provide a simple 
generalization about 
the experience 

or simply begin 
abruptly. 


Organization 


The sequencing of 
events is varied in 
ways that enhances 
the richness of the 
narrative. 


Events are sequenced 
logically with a variety of 
techniques. 


The sequence of events 
is clear. 


Attempts have been 
made to sequence 
events, but control 
is occasionally lost. 


Development 


The use of dialogue 
further enriches 
characterization, and 
detailed description 
and effective 

pacing supports the 
development of the 
narrative. 

The presentation 

of events or 
characterization keeps 
the reader engaged in 
the story. 


Narrative techniques 
such as dialogue, 
description, and pacing 
are used to develop 


events and/or characters. 


Minimal descriptive 
details are used to 
develop characters and/ 
or events; descriptive 
language is rudimentary, 
employs clichés and 
reflects a limited range 
of techniques. 


An attempt may 
have been made to 
develop characters 
and/or events 
through description, 
but the descriptions 
are scant, if present 
at all. 


Vocabulary Figurative language Words, phrases and Some precise language | Everyday language 
and precise detail details that present is used to describe related to characters 
creates vivid images of | a vivid picture of characters and events. and events is used. 
characters and events. | characters and events 

are used. 

Closure The conclusion fulfills | The conclusion is clearly | A closure statement is The paper ends 
the expectations set by | stated and follows presented related to the | abruptly without a 
the introduction and logically from the event | events in the writing. generalization about 
may offer a clever twist | sequence. the experience 
or surprise. or attention to a 

conclusion. 

Other Punctuation and Punctuation and Knowledge of There are 


grammar are used 
competently. 
Sentence structures 
are complex and 
varied. 

Formatting and 
graphics may be 
provided to enhance 
comprehension. 


grammar are used 
competently. Although 
there may be some 
distracting errors, 
meaning is clear. 
Sentence structures 
are for the most part 
complex and varied. 
Formatting and graphics 
may be provided to aid 
comprehension. 


basic punctuation 

and grammar is in 
evidence. However, 
there are frequent and 
significant errors though 
they do not inhibit 
understanding. 

Simple sentences are 
the dominant syntactic 
form used by the writer. 


weaknesses in 
spelling, punctuation 
and grammar, but 
some meaning is 
intelligible. 

Lack of language 
control is evident 
throughout. 
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NCEE asked instructors to provide a range of 
student writing samples, including an A, two Bs and 
one C. When colleges graded the essays differently 
(e.g., high, medium or low), we created a common 
letter scale for all of the essays. The instructors’ 
grades were obscured on the copies of student work 
the panelists used so that their scoring would not be 
influenced by instructor grades. It is worth noting 
that the majority of essays we received from colleges 
were argument or informational, with only one 
college sending narrative essays from an English 
Composition class. 


While there was much agreement between the 
instructors’ and the panelists’ scores at the lower 
end of the grading scale, there was some notable 
disagreement at the mid-range of the scale. Chart 
6 shows the difference in what the panel and the 
instructors considered college ready (i.e., 2.5 on the 


rubrics for the panelists and a GPA of 2.75 for the 
instructors). While 20 percent of essays that the panel 
considered college ready were scored not college ready 
by the instructors, almost half (48 percent) of those 
considered college ready by the instructors were given 
scores of 2 or below by the panel. 


Chart 7 that contrasts instructor grades with 
panel ratings makes this point even more starkly. 
Over seventy-five percent of the essays given a 

B by the instructors were marked 2 or below by 
the panel. This was an issue in particular in the 
argument essays, which often received Bs without 
including well-supported claims. No obvious 
pattern emerged to suggest why the panelists 
graded the writing aspect of the subject matter 
essays more harshly than did the instructors. See 
Example 4 (in Appendix E) for a typical case 

of disagreement. This essay was scored an A by 


CHART 6 Panel Ratings of Essays Scored College Ready and Not College Ready by Instructors 


INSTRUCTOR GRADE 


Not College Ready 


College Ready 


0% 20% 40% 
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Panelist Ratings 


Not College Ready 
i Rating 1 


@ Rating 1.5 
{i Rating 2 
College Ready 
i Rating 2.5 
i Rating 3 
Rating 3.5 


i Rating 4 


60% 80% 


100% 


the instructor and was given a 1.5 by the panel. 
The panelists’ notes suggest that they did not 
believe the author developed an argument. They 
agreed with the instructor that the grammar and 
vocabulary were reasonable. In fact, most of the 
essays where the panel scored the essays lower than 
the instructor involved similar rationales: failure 
of the essay to sufficiently develop the theme or 
argument according to the norms of that genre. It 
seemed that instructors sometimes gave students 

a pass on organization, especially if spelling and 
punctuation were largely correct. It should also be 
noted that there were many examples of agreement 
on the scoring, particularly at the higher scores. See 
Example 5 (in Appendix F) from an English class. 
‘This essay was given an A by the instructor and 3s 
by the panelists. 


We conclude that a significant share of the writing 
that is considered acceptable by community college 
instructors is found wanting on the CCSS-based 
rubric, particularly writing that requires reasoned 
arguments supported by evidence. We also found 
the lack of writing in many of the programs a cause 
for concern. It seems that students are required to 
learn college writing in English Composition and 
then rarely asked to write again. Further, on those 
few occasions when they are asked to write outside 
of English Composition, the standard for what is 
considered acceptable is considerably lower than in 
English Composition. This deprives students of the 
opportunity to build a critical communications skill 
and also suggests that much of the assigned work 
required much less complex analysis and thought than 
might initially be suggested by the tasks at hand. 


CHART 7 The Distribution of Panel Ratings of Student Essays by Instructor Grades 


INSTRUCTOR GRADE 


Panelist Ratings 


Not College Ready 
i Rating 1 


i Rating 1.5 
{i Rating 2 
College Ready 
i Rating 2.5 
Hi Rating 3 
fy Rating 3.5 


i Rating 4 


0% 20% 40% 


60% 


80% 


100% 
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VIT. 


THIS STUDY of initial credit-bearing courses 

in community colleges suggests that only modest 
reading and writing demands are placed on students 
in these courses. While texts assigned include 
content at about an 11'*— 12" grade reading level, 
which is significantly more challenging than 

what they typically encounter in high school, the 
level of processing of those texts required by the 
assigned tasks is, at best, only modestly challenging 
in most courses. The one exception was English 
Composition, where high challenge levels are 
common. Students in the community colleges 

we studied are asked to retrieve information and 
sometimes integrate information from different texts 
in their writing, but only a few courses, outside of 
English Composition classes, ask students to reflect 
on and analyze what they read. 


Reading and understanding technical vocabulary is 
a necessary skill in many of the initial credit bearing 
courses analyzed. While students will not likely 
encounter such vocabulary in high school, experience 
in high school with navigating texts in unfamiliar 
subjects, including texts that contain technical 
vocabulary of some type, would better prepare them 
for the demands of college. Consistent with this 

idea is identification by the CCSSE of reading in 
technical subjects as an important learning objective. 
Consequently, placing some greater emphasis on 
literacy with graphical representations and other 
technical means of communication seems like a 
sensible strategy for high schools to consider. 


English Composition courses in our community 
colleges focus on teaching students the different 
genres of writing needed in college, but many of 

the courses specific to the industry clusters never 
give any writing assignments or assign types of 
writing that might help students develop the writing 
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CONCLUSION 


skills needed for that industry. In addition, far 

too many of the industry classes rely primarily on 
multiple choice tests to assess students’ command 
of knowledge, thus communicating that writing 
ability is not really needed. Aside from sending 

a false signal to students, this shouldn’t be read 

as an excuse for anyone being satisfied with the 
meager amount of writing students are assigned 
in high school. In the first instance, most students 
will be taking an English Composition course and 
not giving adequate attention to writing in high 
school is a recipe for trouble in this course and in 
subsequent college courses students might take. 
Secondly, it is a recipe for trouble in the workforce 
and for participation in civil society. 


For far too many students, the reading and writing 
done in high school do not match either the 
complexity level or the task types they will need in 
their college classes and in many career pathways. 
We found considerable evidence suggesting that 
many of the deficits of secondary school language 
arts instruction are being replicated rather than 
remedied in community college teaching. The 
writing tasks assigned in these community college 
programs are of low challenge, students’ writing 
skills are rarely assessed, and expectations for 
student writing, especially of arguments, are low. 
Our students clearly need better instruction in 
constructing arguments and in laying out their 
thinking logically and persuasively. Such writing 

is at the heart of learning in college to say nothing 
of its essential role in many workplaces. It pushes 
students to gain command of the subjects they are 
studying, to think critically about the strengths and 
weaknesses of different points of view, to anticipate 
counterarguments, and to express their findings 
clearly and persuasively. The target for student 
competence in this aspect of literacy in both our 


high schools and colleges needs to be raised if our 
students are to have a future with promise that they 
all deserve. The call of the Common Core State 
Standards for strengthened instruction in this area is 
a sound first step in this direction. 


The Panel’s concern with community college 
faculties’ low expectations for the literacy levels of 
their students could reasonably be read as a call for 
raising those expectations to much higher levels. 
But we suspect that those expectations are as low as 
they are in part because so many of their incoming 
students cannot achieve even at these low levels. 

To raise the standards in our community colleges 
without concurrently doing what is necessary to 
enable our graduating high school students to meet 
the minimal standards currently in place would be 
irresponsible. Such a policy stance will only make 

a tough situation worse. So action on both fronts 

is clearly urgent. Once the nation has succeeded in 
bringing the literacy levels of our graduating high 
school students up to the current (low) standards of 
our community colleges, we suspect the community 
colleges will be as eager as anyone else to raise those 
standards. But we would hope they would not wait 
for all the evidence to roll in and use the weakness of 
our high schools as an excuse for inaction. 


The issues revealed by this study are clearly 

not limited to the low standards for English 
literacy in our high schools. There is a striking 
mismatch between the kind of English literacy 
skills demanded for success in college and careers 
and the curriculum in our schools. Some of this 
mismatch is addressed by the new Common Core 
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State Standards for English literacy. As such they 
represent a promising first step in righting this ship, 
but their faithful implementation will likely be a 
heavy lift for our schools, and even if successfully 
executed offer no guarantee of fully addressing 

the many shortcomings identified by this study. 
Parallel initiatives on the community college front 
are also in order as is a commitment to build on this 
initial research to deepen our understanding of the 
issues at hand and to track the results of the most 
promising efforts that may be mounted to address 
the shortcomings identified here. 


This report will be jarring for many. Our findings 
paint a very different picture of the actual standards 
for success in our community colleges than many 
have been carrying around in their heads. While 
we are confident that our research techniques have 
enabled us to produce a much more accurate picture 
of those standards than the nation has ever had 
before, we do not regard this report as the last word 
on the subject. We would welcome studies that 
include a much larger random sample of colleges, 
take a closer look at colleges with outstanding 
reputations and gather a larger sample of the 
materials used in courses as well as student work. 
We think it would be worthwhile to do case studies 
of community colleges, looking in more detail at 
classroom practices and interviewing instructors to 
better understand why they are not making full use 
of the texts they assign and gauge their own sense 
of their students’ needs and limitations. It is not 
unusual for researchers, in their reports, to call for 
more research, but we do believe that, in this case, 
more research could pay large dividends. 
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Bachelor of Arts in English and her Master of 
Education, both from the University of Minnesota. 
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John McMillan 


John McMillan is the President of Inquiry By 
Design. He has been involved in school reform 
efforts on a district and national level since 1993. 
He has worked on secondary national literacy 
projects, including the Harvard PACE Project, 

the New Standards Project’s Reference Exam and 
Performance Standards initiatives, and the Ramp 
Up to Literacy and Ramp Up to Advanced Literacy 
programs at the National Center on Education 

and the Economy. Dr. McMillan was a member of 
the Disciplinary Literacy team for English at the 
University of Pittsburgh’s Institute for Learning 
and is currently working with Stanford University 
to develop Common Core State Standards-aligned 
performance assessments for high school English 
Language Arts classrooms. He holds a doctorate in 
English from Texas Christian University, has taught 
at secondary and university levels, and consults 
widely on various literacy-related projects. 


Danielle S. McNamara 


Danielle S. McNamara is a professor in the 
Psychology Department and senior scientist in 

the Learning Sciences Institute at Arizona State 
University. Her academic background includes 

a Linguistics BA (1982), a Clinical Psychology 

MS (1989), and a PhD in Cognitive Psychology 
(1992) from the University of Colorado-Boulder. 

Her research involves the study of learning and 
cognitive sciences and the development of educational 
technologies. The overarching theme of this work is 
to better understand cognitive processes involved in 
memory, knowledge acquisition, reading and writing, 
and to apply that understanding to educational 
practice by creating and testing educational 
technologies (e.g., Coh-Metrix, the iSTART Reading 
Strategy tutor, Writing Pal the writing strategy tutor). 
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A focus of her work is on developing methods 

to improve success for struggling high school 
students. Along those lines, she contributed to 

the writing of the English Language Arts College 
Board Standards for College Success, which had 

a focus on including reading strategies within the 
standards. She has served on the editorial boards 

of Discourse Processes, Memory & Cognition, and 
JEP:LMC and currently serves as associate editor 
for three journals, topiCS, the Cognitive Science 
Journal, and the Journal of Educational Psychology. 
She has served on numerous review panels for the 
Institute of Education Sciences, National Science 
Foundation and National Institute of Child Health 
and Human Development and currently serves on 
the National Institute of Health’s standing review 
panel for cognition and perception. She also serves 
on the governing boards for the Society for Text and 
Discourse and the Cognitive Science Society. 


Sandra Murphy 


Sandra Murphy is professor emerita at the University 
of California, Davis’ School of Education. She has 
written widely on the teaching and assessment of 
writing and has co-authored several books on writing 
assessment, including Designing Writing Tasks for 

the Assessment of Writing (with Leo Ruth), Writing 
Portfolios: A Bridge from Teaching to Assessment 

(with Mary Ann Smith), and Portfolio Practices: 
Lessons from Schools, Districts and States (with Terry 
Underwood). She currently serves as a member of the 
editorial boards of Research on the Teaching of English 
and the Journal of Writing Assessment. Previously she 
served on the advisory board for the Handbook for 
Writing and as a member of the boards of Assessing 
Writing and Educational Assessment. Areas of special 
interest in her research and scholarship include 
writing assessment and its impact on teachers and 
curriculum, reading comprehension and critical 
perspectives on literacy. 
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Dr. Murphy is active as an educational consultant 
who specializes in writing assessment and the 
development of curriculum standards. She co- 
chaired the Steering Committee for the development 
of the new framework for the National Assessment 
of Educational Progress (NAEP) in Writing and 
served on the work group for the development of 
the Common Core State Standards in English 
Language Arts and the Common Core State 
Standards for Literacy in History, Social Studies, 
Science and Technical Writing. She received her 
PhD in Language and Literacy Education from the 
University of California, Berkeley. 


Maricel G. Santos 


Maricel G. Santos is an associate professor of 
English at San Francisco State University, where 
she teaches in the MA in TESOL Program and 
the EdD in Educational Leadership Program. She 
teaches courses in second language acquisition, 
ESL methodology, community-based ESL, 
curriculum development, immigrant literacies, 
and research methods. Dr. Santos also is a research 
scholar supported by a Research Infrastructure in 
Minority Institutions (RIMI) grant from NIH’s 
National Center for Minority Health and Health 
Disparities. Her RIMI research explores adult 
ESL participation as a health-protective factor in 
transnational immigrant communities, as well as 
the ways that adult ESL learners can serve as agents 
of change in health care. In collaboration with the 
California Diabetes Program, she is studying the 
effects of peer support networks and innovative 
curricula on diabetes prevention and awareness 
among beginning-level adult ESL learners (a study 
supported by a 5-year grant from the Centers for 
Disease Control and Prevention). 


Dr. Santos has published a co-edited volume with 
Dr. Kathleen M. Bailey entitled Research on ESL in 
U.S. Community Colleges: People, Programs, and 


Potential (2009, University of Michigan Press). She 
also has authored several adult ESL teacher training 
materials (including study circles for the National 
Center for the Study of Adult Learning and Literacy) 
and articles in Global Health Promotion, Focus 

on Basics, and the Harvard Education Newsletter. 
She serves on the editorial board of the Modern 
Language Journal and previously served on the 
board of the Harvard Educational Review. Prior 

to coming to San Francisco State, she worked on 
various research projects with the National Center 
for the Study of Adult Learning and Literacy. Dr. 
Santos has an EdD in Human Development and 
Psychology, with a focus on Language and Literacy, 
from the Harvard Graduate School of Education, 

an MA in TESOL from the Monterey Institute of 
International Studies, and a BA in English Literature 
from Swarthmore College. 


Howard B. Tinberg 


Howard B. Tinberg is professor of English at Bristol 
Community College in Fall River, Massachusetts, 
where he has taught for more than two decades. His 
major interest over the years has been to promote the 
scholarship of teaching and learning, both within 
English studies and across disciplines. He is the 
author of three books, Border Talk: Writing and 
Knowing at the Two-Year College (NCTE, 1997), 
Writing with Consequence: What Writing Does in the 
Disciplines (Longman, 2003), and (with JP Nadeau) 
Community College Writers: Exceeding Expectations 
(SIU Press, 2010). He is co-editor, with Patrick 
Sullivan, of What is College-Level Writing? (NCTE, 
2006) and, with Patrick Sullivan and Sheridan 

Blau, of What is College-Level Writing? IT (NCTE, 
2010). He is past editor of the journal Teaching 
English in the Two-Year College (published by NCTE). 


Dr. Tinberg has published articles in College English, 
College Composition and Communication, Teaching 
English in the Two-Year College, Pedagogy, ADE 
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Bulletin, English Journal, Change, Journal of Basic 
Writing and Community College Review. He was the 
2004 recipient of the Carnegie/CASE Community 
College Professor of the Year and served as a 
Carnegie Scholar in 2005-2006. He was recently 
elected assistant chair of the Conference on College 
Composition and Communication, the premier 
national organization for college and university 
teachers of rhetoric and composition. He received his 
PhD in English from Brandeis University. 
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With Race to the Top and the rise of the Common 
Core State Standards, issues related to text 
complexity are at the forefront of the national 
conversation about literacy and schooling. The 
pressures of international competition have amplified 
concerns that students across states might not 
experience equitable and rigorous literacy tasks 
featuring texts that are comparably difficult. 
Unfortunately, consensus determinations about levels 
of complexity in texts are difficult to make. The 
world of readability is a highly contested one so there 
are a large number of disparate tools available to 
schools, teachers, publishers, and testing companies 
interested in assessing difficulty or, conversely, 
crafting texts for specific ages or skill levels. 


Almost all readability metrics insist on reducing 
measures of difficulty to a single score, one that 
accounts for some combination of vocabulary 
difficulty, syntactic complexity, and text structure 
and that purports to be useful across genres and 

for both literary and informational texts. The 
consequences of this are an inevitable reductionism 
because single score metrics inevitably oversimplify 
the nature and dimensions of difficulty. The classic 
example of this is found in the way most readability 
measures overpredict the difficulty of informational 
texts while underpredicting the difficulty of narrative 
or literary texts. 


‘This study was undertaken to inform the work the 
National Center on Education and the Economy is 
conducting with states in the Consortium on Board 
Examination Systems (SCOBES). These states— 
which include Arizona, Connecticut, New Mexico, 
New York, Kentucky, Maine, Vermont, and New 
Hampshire—indicated strong interest in piloting 
board exams in their high schools. Related to the 
pilot, NCEE has assembled a technical advisory 
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committee whose work it will be to recommend 

cut scores on those examinations that will certify 
readiness for entry-level credit courses at open- 
admissions postsecondary institutions in the COBES 
states. To inform that work, we set out to conduct 

a survey of entry-level textbooks collected from 
entry level credit courses at a random set of open 
admissions postsecondary institutions in the eight 
COBES states. These were gathered from eight 
popular and diverse program areas. We set out to 
learn something about the kinds and levels of text 
complexity in-coming college students are likely 

to encounter in entry-level courses in an effort to 
determine where high school students need to be in 
terms of their ability to deal with levels and kinds of 
text complexity. 


To accomplish this, we aimed to apply—and 
compare across—a core set of readability 
measurement tools that are widely considered as best 
correlating to scores on comprehension tests!. Our 
objective was to use this set of readability metrics 

to analyze and evaluate a set of entry-level post- 
secondary textbooks in order to inform the current 
conversation about the skill set and experiences high 
school students need to successfully transition from 
secondary to post-secondary education. 


Determining the complexity levels of a large set 

of texts is no easy matter. Pinpointing the exact 
challenge level of materials necessary for a student 
to be successful in a high school biology or literature 
course or in an entry-level course at a technical 
school—for example, in an IT course or a class on 
automotive repair—requires negotiating several 
problematic factors: 


| Different tests measure comprehension in different ways, though 
all measure subskills that correlate with comprehension. Measur- 
ing subskills is not the same as measuring comprehension itself. 
This is a critical point in the conversation about text complexity 
and readability. 


e ‘The specific constructs different readability 
formulas measure; 


e Discrepancies, the result of different definitions 
or conceptions of seemingly straightforward 
constructs such as “vocabulary” and “sentence 
length,” that account for differences between 
formulas measuring the same passages; 


e Determining which among the myriad of 
readability metrics currently represent the gold 
standards in measurement and why these measures 
are granted this status, including how they 
correlate with tests of reading comprehension; 


e How non-prose text elements such as maps, 
tables, and graphs (referred to as “documents” 
by Mosenthal and Kirsch) influence complexity 
and are accounted for by readability metrics. 


On Readability Formulas 


Critics of readability formulas often point to the 
discrepancies that exist between the disparate scores 
different readability formulas assign to the same 
texts. They contend that such discrepancies are 
indications of the lack of precision of the formulas. 
Critics also point out that these measures only take 
into account the surface features of a text, that they 
ignore features such as content and organization. 


Defenders of readability formulas contend that it 

is not how the instruments agree or disagree with 
one another on a particular text, but their degree of 
consistency in predicting difficulty over a range of 
graded texts. And that while readability measures do 
largely focus on surface features such as vocabulary 
and sentence length, these features have been shown to 
be the best predictors of text difficulty as measured by 
reading comprehension measures. 
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In either case, the range of scores provided by 
different formulas remind us that they are not 
perfect predictors. They provide probability 
statements or, rather, rough estimates of text 
difficulty that, in most cases, correlate closely with 
one another. All readability formulas primarily 
account for variations in vocabulary, syntax, and 
text structure. The problem is that these formulas 
measure these constructs in often very different 
ways. Some examples: 


e A measurement of the vocabulary challenges 
posed by a given text might address one or 
more of the following things and define it as 
“vocabulary”: word familiarity, word difficulty, 
number of letters per word, or the number of 
syllables per word. 


e Measures of syntactic complexity might count 
the number of words per sentence, the number 
of words prior to the verb in the main clause, 
or at levels of cohesion in a text as indicated 
by cohesive links in a text, e.g. conjunctions, 
adversatives, and other transition words. 


e Accounting for the difficulties posed by 
different text structures might address the 
predictability or accessibility of different 
narrative and nonnarrative structures, including 
the way authors indicate relationships between 
parts and ideas in a text and the degree to 
which those relationships. 


Perhaps the most highly regarded tool for evaluating 
the difficulty level of texts for assisted instructional 
environments (e.g. a secondary school classroom) is 
the Qualitative Assessment of Text Complexity created 
by Jeanne Chall, Glenda Bissex, Sue Conrad, and 
Susan Harris-Sharples. This assessment supplies 
readers with graded passages from a range of 
published works, as well as a method for matching 
texts to those passages in order to rank texts by 
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difficulty. After training and calibrating, judges 
complete worksheets as they match new passages to 
those in the assessment book. There are a total of 52 
passages in the QATD that are arranged by grade- 
level across the following six “scales”: 

e Literature 

e Popular fiction 

e Life Science 

e Physical science 

e Narrative Social Studies 

e Expository Social Studies. 
The authors of the QATD selected passages for 
each scale based on the following grade-related 
requirements for the reader: 

¢ Knowledge of vocabulary, 

e Familiarity with sentence structure, 

e Subject related and cultural knowledge, 

e Technical knowledge, 

e Density of ideas, and 

¢ Level of reasoning. 
Hence, the QATD is informed by, but goes well 
beyond, computer-based readability formulas 
although it does take into account both vocabulary 
and sentence length. For this reason, even though 
it is limited to six text type categories and a narrow 
band of content areas, it is widely regarded as the 


gold standard in the readability world. 


William DuBay, in his book Smart Language: 
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Readers, Readability, and the Grading of Text, used 
the QATD passages to determine the overall validity 
of various readability formulas. To do this, he used 
the computer program “Readability Calculations,” 
available from Micro Light and Power at 
http://www.micropowerandlight.com and the 52 
normed passages in the Qualitative Assessment of 
Text Difficulty tool. The results listed here show the 
correlations between the general-purpose readability 


formulas and the normed passages: 


Formula Correlation Standard Error 
Dale-Chall ag 3 1.76 
Flesch-Kincaid se) 1.9 
Gunning Fog 91 2.00 
el 88 2.28 
ee Reading 88 2 AA 
Fry Graph .86 Zo 
FORCAST .66 3.61 


To appreciate the work undertaken to create a valid 
readability measure, consider DuBay’s account of 
the original Dale-Chall formula (1948), DuBay’s 
strongest correlator and widely regarded as 

perhaps the most consistent and valid formula for 
determining readability: 


The original Dale-Chall formula uses a sentence 
length variable plus a percentage of “hard 
words” not found in Dale’s long list of 3,000 
easy words. It was updated in 1995 (the New 
Dale-Chall) and was validated against a variety 
of criteria, including the following: 


e 32 passages tested by Bormuth (1971) on 4th to 
12th-grade students. 


° 36 passages tested by Miller and Coleman 
(1967) on 479 college students. 


e 80 passages tested by MacGinitie and Tretiak 
(1971) on college and graduate students. 


e 12 technical passages tested by Caylor et al. 
(1973) on 395 Air Force trainees. 


The new version of the Dale-Chall formula (1995) 


was also cross-validated with: 
e The Gates-MacGinitie Reading Test. 


e The Diagnostic Assessments of Reading and 
Trial Teaching Strategies (DARTTS). 


e The National Assessment of Reading Progress. 
e ‘The Spache Formula. 
e The Fry Graph. 


e Average judgments of teachers on the reading 
level of 50 passages of literature. 


The new formula correlates .92 with the Bormuth 
Mean Cloze Scores, making it the most valid of the 


popular formulas. (Dubay 94-95) 


What We Did 


For this project, we analyzed and evaluated passages 
selected from 86 textbooks across a selection of 
popular areas of study used in entry-level courses 

in eight community colleges and open admissions 
universities. The areas of study were: 
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e Nursing 

e Business/Accounting 

e Automotive Technology 

e Information Technology 

e Criminal Justice 

e Early Childhood Education 

e Biotech/Electrical Technology 


In addition to these textbooks, we also looked at 
textbooks used in required initial mathematics and 
English Composition courses required for these 
programs. To evaluate the difficulty levels posed by 
these texts, we used the Qualitative Assessment of 
Text Difficulty and the three readability programs 
that, according to William DuBay, correlated most 
strongly with the QATD scale passages: 


e The New Dale-Chall 
e The Gunning-Fog 
e Flesch Kincaid 


We selected passages from each textbook for scoring 
in the following way: From each book, five 100 word 
passages were selected at random approximately one 
every 100 pages. Ten selections were chosen from 
the composition textbooks: five passages that gave 
directions or provided commentary and five that 
contained model texts for different writing types 
featured in a particular textbook. ‘The rationale for 
sampling composition textbooks in this way was 
simple: directions and commentary in a composition 
textbook are ubiquitous and yet substantially 
different from the genre models students are likely to 
encounter elsewhere in a reader. 
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Passages were and saved in Microsoft Word and plain 
text formats. The Word documents were used for the 
QATD scoring and for the reporting of the scores 
after the readability formulas (New Dale-Chall, Flesch 
Kincaid, and Gunning-Fog) were run. Plain text files 
were required for running the readability formulas. 
For each readability formula, the five passages for 
each textbook were scored in aggregate, providing a 
single score that accounted for all the selections in a 
given text. We ran all 86 textbooks through the three 
computer-based readability programs. 


Next, we created a subset of 46 textbooks that 
mapped to the scale passages in the QATD tool. 
(The remaining 40 texts—for example, mathematics 
and Information Technology texts—did not 
correlate closely enough to any of the QATD scales 
and so were not included in this portion of the 
study.) This subset included humanities and sciences 
texts and textbooks from other disciplines that 
utilized text structures and conceptual taxonomies 
similar to those in the life and physical sciences 
scale passages. For example, automotive technology 
textbooks organized information in the domain 

in connected sequences according to function and 
relationships between systems in ways quite similar 
to biology or anatomy; hence, they were scored using 
the QATD scale for Life Sciences where knowledge 


was similarly structured by system and function. 


Next, we convened a QATD scoring session with 
group of educators with secondary and post- 
secondary experience’. The group went through an 
initial training and calibration seminar that focused 
on a shared set of texts and scales. After calibration, 


? QATD scoring committee: Sally Hampton (America’s Choice), John 
McMillan (Inquiry By Design), Christi Duque (Instructor, Tarrant 
County College, Fort Worth, TX), Manuel Alvear (High School 
English teacher, Country Day School, Fort Worth, TX), Adrianne 
Richardson (Instructional Coach, Fort Worth ISD), Faye Richardson 
(Secondary English and Reading teacher, Fort Worth ISD), Aimee 
Shimamoto Hunter (educational consultant), and Bonnie Dickinson 
(America’s Choice). 
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the large group separated into three smaller groups 
and scored the remaining texts. 


The Qualitative Assessment 
of Text Difficulty 


The QATD is a judgment-based tool used for 
estimating reading difficulty. It was created by 
Jeanne Chall and three of her doctoral students: 
Glenda Bissex, Sue Conard, and Susan Harris- 
Sharples. The QATD was written during a time, 
like now, in which there was renewed enthusiasm 
for holistic scoring to assess student writing and to 
estimate the readability of texts. 


The QATD presents “a method for the qualitative 
assessment of text difficulty—a method that relies 
on total impression rather than on an analysis of text 
features” (1). The QATD method involves mapping 
passages selected from texts to sets of exemplar 
passages that have been scaled for comprehension 
difficulty. The exemplar passages are sorted into six 
types: Literature, Popular Fiction, Life Sciences, 
Physical Sciences, Narrative Social Studies, and 
Expository Social Studies. The QATD provides scale 
passages for each type rated by grade level from 1* 
to 16+ (graduate school). The following criteria were 
used to scale each passage: 


Vocabulary (general and content-related) 
e Sentence structure 


e Prior knowledge of general and subject-related 
concepts, including life experiences and 
cultural/literary knowledge for the fiction scales. 


e For the sciences and social studies scales, 
attention was also paid to the conceptual load 
or density of ideas within a passage, as well 
as the level of reasoning (including drawing 
inferences and a sense of how knowledge is 


constructed in a discipline) required for a reader 
to comprehend the text. 


To use the QATD, a reader or group of readers 
samples representative passages from a text they are 
evaluating and compares them to the scale passages 
in the QATD. To do this, a reader first makes a 
decision regarding the content of the text: Is it 
literature? Popular fiction? Narrative Social Studies? 
Expository Social Studies? If it is in the sciences, 

is it a life science or a physical science? Once a 
determination is made, the reader take selections 

of approximately 100 words from the text she is 
evaluating and compares it to the scale passages in 
the QATD®. In making this comparison, the reader 
looks for similarities in the challenges posed by 
vocabulary, the length and structures of sentences 
featured in the text sample and the scale passage, and 
the level of reasoning required to comprehend the 
text. For example, here are two scale passages from 


the Life Sciences category of the QATD: 


LEVEL 2 


Frogs and toads are amphibians without tails. 


You can tell frogs apart by the pattern on 
their skins. 


Some frogs have stripes. 


The Swamp Tree frog has dark stripes down 
its back. 


3 For this study, we took five 100 word samples from each text, the 
first near the beginning of the book (within the first 100 pages) 
and the remaining four passages approximately every 100 pages 
thereafter. This was a necessary variation on the QATD protocol 
(given the number and size of the texts in the study) which sug- 
gests that “for books of 150 pages or longer, select one sample 
from every 50th page. The first sample should be taken from the 
beginning section of the book (but not the first page), and system- 
atically thereafter, from every 50th page” (15). 
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‘The Green Tree frog has a light stripe down 


each side and along its legs. 


‘The Sheep frog has a light stripe down the 
middle of the back. 


Which is which? 
Sometimes size is a clue. 
‘The Bullfrog is big. It can be 8 inches long. 


‘The Green frog is smaller. It gets to be only 
3% inches long. 


LEVEL 11-12 


Frogs and toad have an aquatic larval stage, 
the familiar tadpole. The fishlike tadpole has 
gills which are later lost in metamorphosis. 
The moist skin of frogs and other amphibians 
contains mucous glands that assist in 
maintaining the moisture. Moreover, the eggs 
of amphibians, laid in water or other moist 
areas, are usually covered with a gelatinous 
substance. Thus amphibians remain dependent 
on aquatic (or at least wet) environments in 
many ways. 


This group also shows adaptations for living 
on land. Most importantly, adults have 

lungs adapted for air breathing and are 
therefore no longer dependent on water for 
gas exchange. (It can occur through the skin 
when amphibians are in water.) Furthermore, 
the two nostrils are connected to the mouth 
cavity to facilitate breathing through the 
lungs. Almost all amphibians have two pair 
of jointed appendages that permit locomotion 
both on land and in water. Frogs and toads 
also have sound-sensitive membranes (“external 
eardrums”) on their bodies; such specialized 
sense organs are essential for land dwellers, 
because air does not transmit sound waves as 
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efficiently as water. Finally, amphibians have a 
more efficient type of circulatory system than 
fish, including a heart with three chambers 
rather than two. 


Here is a selection from Inquiry Into Life by Sylvia S. 
Mader, a textbook we evaluated for the project that 
readers determined best matched the Life Sciences 
11-12 scale passage: 


A covalent bond results when two atoms 

share electrons in such a way that each atom 
has an octet of electrons in the outer shell. In 

a hydrogen atom, the outer shell is complete 
when it contains two electrons. If hydrogen is 
in the presence of a strong electron acceptor, 

it gives up its electron to become a hydrogen 
ion (H+). But if this is not possible, hydrogen 
can share with another atom and thereby 

have a completed outer shell. For example, 

one hydrogen atom will share with another 
hydrogen atom. Their two orbitals overlap, and 
the electrons are shared between them. Because 
they share the electron pair, each atom has a 
completed outer shell. (25) 


Readers of this passage determined that the passages 
were comparable in terms of: 


e Sentences: sentences in both passages were 
similar in length and complexity. 


e Vocabulary: passages contained comparable 
amounts of difficult words and 
specialized terminology 


e Level of reasoning: required reader to make 
connections between ideas that were not always 
explicit. For example, the lack of cohesive 
links between first two sentences in Inquiry 
Into Life is challenging because a reader must 
determine that sentences two through seven are 
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illustrating scenarios for the creation of covalent 
bonds using the hydrogen atom as a case. 


e Comparable levels of idea density with similar 
levels of support through transition words. The 
shifting in Mader’s text (“if this, then this,” 
“but if this (then) this” maps to signpost words 
in the scale passage (“moreover,” “furthermore,” 
“finally”). In both passages a reader must 
use these words to track connections and 
relationships between sentences and ideas in order 
to grasp the overall coherence of the selections. 


One of the challenges we faced in this project was 
created by the limited number of scale types in the 
QATD. After all, we were looking at textbooks from 
automotive mechanics, accounting, and criminal 
justice, as well as at textbooks from disciplines that 
mapped directly to the scale content categories: 
biology, narrative passages from composition readers, 
and selections from history textbooks. To use the 
QATD across a wider range of content areas, we 
made judgments about texts according to the ways 
we observed them organizing knowledge in a field. 
For example, we used the life sciences scale to score 
passages from automotive mechanics because both 
were “descriptive/technical,” primarily factual and 
at the highest levels of difficulty becoming more 
microscopic, detailed, and technical (QATD 60). 
Indeed, the challenges posed by a college-level 
biology textbook and an automotive mechanics 

text used in a community college are in many ways 
similar: seemingly massive amounts of information, 
often organized by systems and in hierarchies, using 
precise, technical vocabulary and featuring many, 
often highly detailed visuals that supplement the 
print text. Here is a selection from Stockel, Stockel, 
and Johanson’s Auto Fundamentals, a text that 
readers scored at 11-12 using the Life Sciences scale: 


Air operated shock absorbers have hydraulic 
dampening systems which operate in the same 


manner as those on conventional shocks. In 
addition, they contain a sealed air chamber, 
which is acted on by pressure from a height 
control sensor. Varying the pressure to the air 
chamber causes the air shock to increase or 
decrease its length or operating range. 


Air pressure is delivered to the air shocks 
through plastic tubing. The tubing connects the 
shocks to an air valve. Air pressure for raising 
the shocks is generally obtained from an outside 
source, such as a service station compressor, and 
is admitted through the air valve. To deplete 
the shocks of unwanted air (lower vehicle curb 
height), the air valve core is depressed, allowing 


air to escape. (459) 


Like the scale passage and the selection from Mader’s 
Inquiry Into Life described above, this passage also 
features sentences of relatively long and complex 
construction and a large number of specialized, 
discipline-specific vocabulary. The passage is dense 
with ideas and provides some support for readers 
through words and phrases such as “which” and “in 
addition,” though the burden is largely on the reader to 
make clear the implicit connections between sentences. 


Content Area Findings 


On the whole, all of the textbooks we reviewed 
present significant challenges for students using them 
in /argely unassisted environments. Among the greatest 
challenges is the development of the capacity to 
comprehend and process large amounts of declarative 
knowledge that very often appear to have distant and 
unclear connections to real contexts. Summaries of 
the findings for each discipline are provided below. 
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Nursing 


11-12 13-15 (14.075) 2°99 15.76 


Nursing textbooks were among the most difficult texts 
according to all of the complexity metrics. Compared to 
texts in the other disciplines, nursing texts featured 
perhaps the widest range of text types, including 
documents (e.g. table and graphs), procedural 
passages, technical passages featuring dense and 
specialized concept loads, as well as descriptive 
passages featuring non-technical vocabulary. 
Technical passages also featured complex syntactic 
structures that also significantly escalate difficulty 
levels for readers. The three samples below (from 
Fundamentals of Nursing, by Harkreader, Hogan, 
and Thobaben) provide glimpses of procedural, 


technical, and descriptive (non-technical) passages: 


Procedural 


Preventing needlestick injuries is a vital concern 
in the health care industry. The Occupational 
Safety and Health Administration (OSHA) 
mandates the use of standard precautions, 
which include proper handling and disposal 

of sharp instruments including needles. After 
administering an injection, discard the syringe 
and needle, without recapping the needle, in 

a rigid container that has been specifically 
labeled and provided for that purpose (Figure 
21-8). The container should be leakproof and 
puncture-proof. Rigid containers are kept on 
medication cards in all client care settings 

and are wall-mounted in client rooms and 
treatment areas in many acute care institutions. 
To prevent the risk of needlestick injury, do not 
try to place a used syringe into an overfilled 
container. (459) 


43 


B Text Complexity Report 


Technical 


The electrolytes that are most plentiful inside 
the cells are potassium, magnesium, phosphate, 
and protein. Sodium, calcium, chloride, 
hydrogen, and bicarbonate are the most 
plentiful electrolytes in the extracellular fluid. 
Electrolytes exert a major influence on the 
movement of water between compartments, on 
enzyme reactions, on neuromuscular activity, 
and on acid-base regulation. The specific 
functions of protein, hydrogen, bicarbonate, 
and other electrolytes as they affect acid-base 
balance will be discussed later. 


It is through complex regulatory systems 
that the body maintains electrical neutrality. 
This means that the number of negative ions 
(anions) equals the number of positive ions 
(cations) in the body. Table 26-1 lists the 
functions and regulators of ions important in 


the body. (615) 


Descriptive non-technical 


Personal hygiene consists of those activities 
that an individual undertakes to maintain 
body cleanliness and freedom from disease— 
namely bathing, oral hygiene, and hair care. It 
is essential to both physical and psychological 
well-being that hygiene needs be adequately 
met. If the body is not kept clean, the skin is 
compromised and the body can be threatened 
by infection or disease. The individual’s 
comfort, self-esteem, and body image are also 
enhanced by cleanliness of the body. When 


illness or injury prevents the client from meeting 


self-care needs, it is your responsibility to assist 
clients in a culturally sensitive manner to be 

as independent as possible, and to provide 
clients with information and resources needed 
to resume self-care abilities to the extent that 
physical and mental capacities allow. (785) 
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Early Childhood Education 


9-10 9-10 (9.78) 


11.87 14.24 


Early childhood textbooks were among the least difficult of 
the texts scored. Concept loads were generally minimal 
in comparison to other disciplines that featured often 
significantly greater amounts of specialized vocabulary 
and concepts. Documents in early childhood textbooks 
were mostly simple lists or maps that lacked embedded 
elements and which required very little if any outside 
knowledge for comprehension. 


Below is a sample passage from Beginning Essentials 
in Early Childhood Education, by Gordon and 
Browne. This selection exemplifies some of the 
typical characteristics of these texts: relatively 
uncomplicated sentence structures, low concept 
loads, and non-technical vocabulary (not to mention 
broad and suspect generalizations): 


Gifted children have long attention spans, learn 
rapidly, and have good memories and advanced 
vocabularies. They ask a lot of questions and are 
able to express their ideas easily. Independent 
and imaginative, gifted children may be bored 
by normal activities. Socially, the gifted child is 
sought after by peers, yet may be uneasy about 
relationships with other children. (109) 


Automotive Technology 


11-12 11-12 9.54 E77. 


Although not the most difficult according to the 
computer-based programs, Auto texts were considered 
to be among the most challenging by QATD judges 

who all agreed that prior knowledge was essential 

for comprehension given the amount of specialized 
vocabulary and the sheer amount of systems-specific 
information that accumulates across the different texts. 
Two of the three computer-based metrics, as well as 
the QATD scoring group, scored these texts as upper 
high school level texts—though it is important to 
note that none of these metrics accounted for the 
frequency and complexity of the “documents” (e.g. 
maps, charts, graphs) included in these texts. Our 
survey of documents using the PMOSE/IKIRSCH 
tool rated the documents in automotive technology 
texts as the most complex across all the disciplines, a 
finding that suggests that these are college-level texts 
that pose significant challenges for readers. Below are 
sample passages from automotive texts that illustrate 
different characteristics of these texts: 


Highly specialized and dense vocabulary loads 


Diodes are used where alternating current (ac) 
must be rectified (changed) to direct current 
(dc). They are used in alternators, which charges 
the battery and operates other vehicle systems. 
Alternator diodes are arranged to permit the 
current to leave the alternator in one direction 
only. Modern alternators contain six diodes 

to allow all of the alternator output to be 
processed into direct current. Diodes are also 
used in air conditioning compressor and other 
vehicle circuits. 
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Another type of diode is the Zener diode, 
which will not allow current to flow until a 
certain voltage is reached. When the triggering 
voltage is reached, the diode will become a 
conductor, allowing current to pass. Zener 
diodes are often used in electronic voltage 
regulators. (Stockel at al, 117) 


Vocabulary loads are sometimes mitigated by simpler 
sentence structures 


Air operated shock absorbers have hydraulic 
dampening systems which operate in the same 
manner as those on conventional shocks. In 
addition, they contain a sealed air chamber, 
which is acted on by pressure from a height 
control sensor. Varying the pressure to the air 
chamber causes the air shock to increase or 
decrease its length or operating range. 


Air pressure is delivered to the air shocks 
through plastic tubing. The tubing connects the 
shocks to an air valve. Air pressure for raising 
the shocks is generally obtained from an outside 
source, such as a service station compressor, and 
is admitted through the air valve. To deplete 
the shocks of unwanted air (lower vehicle curb 
height), the air valve core is depressed, allowing 
air to escape. (Stockel et al, 459) 


Descriptive passages that assume prior knowledge 
for comprehension 


On older pressurized cooling systems, the 
cooling system could never by completely filled, 
since coolant expelled through the pressure 

cap was replaced by air when the engine cooled 
off. This meant that the cooling system could 
never be operated at full capacity. The closed 
cooling system, which had the addition of a 
coolant recovery system, provided vehicles 

with a cooling system that could be operated at 
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full capacity. The major features are a coolant 
reservoir and a different pressure cap design. 
Some systems use the coolant reservoir as part 
of the actual cooling system. The pressure caps 
in these systems are mounted on the reservoir 


itself. (Stockel et al, 203) 


Criminal Justice 


11-12 (10.5) 13-15 (14.08) 14.5 


17.17 


Overall, the Criminal Justice texts were identified as 
the most challenging by the computer-based metrics. 
QATD judges did not agree with this assessment, 
although the group still scored these texts at upper high 
school levels. Difficult vocabulary and significant 
amounts of discipline-specific concepts (e.g. see 
below: “precedent,” “applied,” “mala prohibitum,” 
and “mala in se”) characterize these texts, though 


» 


textbook authors often made efforts to define some 
of these terms in context. Criminal Justice texts also 
frequently featured long sentences with sophisticated 
syntactic structures (for an example, see the last 
sentence in the passage below). 


‘The present English system of law came into 
existence during the reign of Henry II (1154- 
1189), when royal judges began to publish their 
decisions in local cases. This allowed judicial 
precedents to be established and a national 

law to accumulate. Other judges began to 

use these written decisions as a basis for their 
decision making, and eventually a fixed body of 
legal rules and principles emerged. If the new 
rules were successfully applied in a number of 
different cases, they would become precedents, 
which would then be commonly applied in all 
similar cases. This unified system evolved into 
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a common law of the country that incorporated 
local custom and practice into a national code. 
Crimes that were mala in se, inherently evil and 
depraved (such as murder, burglary and arson), 
and were the cornerstone of the common law, 
were joined by new mala prohibitum crimes 
such as embezzlement, which reflected existing 
social and economic conditions. (Siegel 88) 


Information Technology 


n/a 11-12 (12) 


10.99 


13.54 


‘The IT texts were not scored using the QATD tool 
because the judges agreed that the texts did not map 
well to any of the scale passages used in the QATD tool. 


Like the automotive technology texts, the IT 
textbooks were marked by wide use of documents 
(e.g. charts, maps, graphs, and annotated 
screenshots) that required significant amounts of 
prior knowledge to comprehend. 


Only a small number of operations, such as 
arithmetic and assignment operations, are 
explicitly defined in C + +. Many of the 
functions and symbols needed to run a C 

+ + program are provided as a collection of 
libraries. Every library has a name and is 
referred to by a header file. For example, the 
descriptions of the functions needed to perform 
input/output (I/O) are contained in the header 
file iostream. Similarly, the descriptions of 
some very useful mathematical functions, such 
as power, absolute, and sine, are contained 

in the header file cmath. If you want to use 

I/O or math functions, you need to tell the 
computer where to find the necessary code. You 


use preprocessor directives and the names of 
header files to tell the computer the locations 
of the code provided in libraries. Preprocessor 
directives are processed by a program called a 


preprocessor. (Malik, 75) 


Business/Accounting 


11-12 11-12 (11.86) 11.8 14.7 


Sometimes dense and marked by very specialized 
language, difficulties in these texts were occasionally 
ameliorated by authors intentional use of examples 
and anecdotes to illustrate difficult concepts. Even 

so, selections such as those below from Weygandt, 
Kieso, and Kimmel’s Accounting Principles and Rich, 
Jones, Heitger, Mowen, and Hansen’s Cornerstones 
of Financial and Managerial Accounting still required 
significant amounts of background and concept 
knowledge for comprehension. 


When the seller elects not to offer a cash 
discount for prompt payment, credit terms 

will specify only the maximum time period for 
paying the balance due. For example, the invoice 
may state the time period as n/30, n/60, or n/10 
EOM. This means, respectively, that the buyer 
must pay the net amount in 30 days, 60 days, or 
within the first 10 days of the next month. 


When the buyer pays an invoice within the 
discount period, the amount of the discount 
decreases Merchandise Inventory. Why? 
Because companies record inventory at cost 
and, by paying within the discount period, 
the merchandiser has reduced that cost. To 
illustrate, assume Sauk Stereo pays the balance 
due of $3,500 (gross invoice price of $3,800 
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less purchase returns and allowances of $300) 
on May 14, the last day of the discount period. 
The cash discount is $70 ($3,500 x 2%), and 
Sauk Stereo pays $3,430 ($3,500 - $70). The 
entry Sauk makes to record its May 14 payment 
decreases (debits) Accounts Payable by the 
amount of the gross invoice price, reduces (credits) 
Merchandise Inventory by the $70 discount, and 
reduces (credits) Cash by the net amount owed. 
(Weygandt, Kieso, and Kimmel 200) 


In addition, the trial balance is used to prove 
the equality of debits and credits. If debits did 
not equal credits, the accountant would quickly 
know that an error had been made. The error 
could have been in the journalizing of the 
transaction, the posting of the transaction, or 
in the computation of the balance in the ledger. 
However, a word of caution is necessary here: A 
trial balance whose debits equal credits does 
not mean that all transactions were recorded 
correctly. A trial balance will not detect errors 
of analysis or amounts. Sometimes the wrong 
account is selected for a journal entry or an 
incorrect amount is recorded for a transaction. 
In other cases, a journal entry is omitted or 
entered twice. As long as both the debit and 
credit portions of the journal entry or posting 
reflect the incorrect information, the debit and 
credit totals in a trial balance will be equal. 
(Rich, Jones, Heitger, Mowen, and Hansen 80) 
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Biotech/Electrical Technology 


Mathematics (including Statistics) 


13-15 13-15 (12.58) 11.46 14.7 


S292 


n/a 11-12 (10.69) 10.11 


‘This small group of texts was comprised of only 
three texts: Thinking Like an Engineer by Gerrish, 
Dugger, and Roberts, Electric Circuits by Nahvi and 
Edminister, and Electricity and Electronics by Stephan, 
Bowman, Park, Sill and Ohland. All metrics scored 
these texts at the upper levels of difficulty. QATD 
scores of these texts marked them as the most challenging 
of all the disciplines. Passages (see below for a sample 
from Electricity and Electronics by Gerrish, Dugger, 
and Roberts) from these texts were marked by 
content-specific vocabulary and conceptually dense 
passages that sometimes featured shorter sentences 
that while perhaps offsetting some of the difficulty, 
still posed—according to three of the four metrics— 
post-high school level challenges for readers. 


A ground fault interrupter provides protection by 
monitoring and comparing the current through 
the hot and neutral conductors. A complete 
circuit has the same current in the hot and 
neutral conductors. If a ground fault occurs, part 
of the current will flow to ground. When part 
of the current flows to ground, the comparator 
circuit detects an unbalanced condition between 
the hot and neutral currents. If the difference 
between the hot and neutral conductor exceeds 

5 mA, the comparator circuit will energize the 
trip coil and cause a contact to open in the hot 
conductor circuit. This stops the flow of current 
through both the outlet and the person holding 
a device that is plugged into the outlet. Electric 
shock is stopped almost instantaneously. After 
the GF is tripped, the red reset button needs to 
be pressed to reset the GFI trip mechanism once 
more. See Figure 2-22. Be aware of a condition 
known as nuisance trip, which occurs when there 
is excessive moisture in the device area. (47) 
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The mathematics texts were not scored using the 
QATD tool because the texts did not map well to 
any of the scale passages used in the QATD tool. 

The New Dale Chall and Flesch Kincaid indicated that 
on average the passages in math texts were in the upper 
high school range. Consistent with scoring in the other 
disciplines, the Gunning Fog rated them higher. 


The connection with the first interpretation is 
that if we sketch the curve y = f(x), then the 
instantaneous rate of change is the slope of the 
tangent to this curve at the point where x = a. 
This means that when the derivative is large 
(and therefore the curve is steep, as at the point 
P in Figure 12), the y-values change rapidly. 
When the derivative is small, the curve is 
relatively flat and the y-values change slowly. 


In particular, ifs = f(t) is the position function 
of a particle that moves along a straight 

line, then f“(a) is the rate of change of the 
displacement s with respect to the time ¢. In 
other words, f (a) is the velocity of the particle 
at time t = a. The speed of the particle is the 
absolute value of the velocity, that is, | f(a). 


In the following example we estimate the rate 
of change of the national debt with respect to 
time. Here the function is defined not by a 

formula but by a table of values. (Stewart 79) 


English Composition 


NDC 
Composition— 9-10 
Anthologies 9-10 (9.5) 10.36 | 13.02 


NDC 
Composition 11-12 
onesie 9-10 (10.85) 11.58 | 13.86 


Passages in composition textbooks were sorted into 
two kinds: anthologies and texts. This distinction 
seemed important given the widely different function 
of these passages in the books. “Anthologies” are 
those model texts that the authors selected as 
examples of writing of the different genre students 
were expected to write in in a composition course. 
‘These texts covered a wide range of narrative and 
nonnarrative genre. 


“Texts” were those passages composed by 

the textbook authors to introduce or provide 
commentary or advice on writing in different genre. 
‘These passages frequently had an informal tone and 
were marked by a “how-to” spirit intended to support 
students understanding and work in a particular 
form. The two passages below are from Lunsford, 
Ruszkiewicz, and Walters’ Everything’s an Argument. 
On the whole, composition textbooks were among the 
least difficult of those reviewed in this study. 


Models 


One hundred and twenty correctional 

officers at the Louisiana State Penitentiary at 
Angola and Holman State Prison in Alabama 
were interviewed anonymously in order to 
understand broad areas of the execution 
process. The one to two hour interviews were 
conducted over the summers of 2000 and 2001. 
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During 2000, interviews were conducted of 
fifty of fifty-two members of the Louisiana 
execution team. During 2001, fifty interviews 
were conducted of security officers who 

either work on Death Row or a part of the 
execution process in Louisiana. An additional 
twenty interviews were carried out involving 
correctional officers who have worked with 
executions in Alabama. In addition to gathering 
demographic and background information, 

a number of questions were asked about the 
following topics: (1) The execution experience, 
including roles, reactions, preparations, 
emotions experienced, and changes over time; 
(2) Stresses related to their job and methods 

to cope with stress; (3) Support network 

and influence of work on relationships; (4) 
Aftermath of execution experience for the 
officer. Based on our interviews, we were able 
to recreate the step-by-step process of carrying 
out an execution. The process was largely 
similar in the two states, but differed due to 
both situational factors with the two facilities 
as well as the mode of execution employed in 
each state. (Louisiana uses lethal injection while 
Alabama is one of two remaining states still 
employing the electric chair as its sole means of 
execution.) (205) 


Directions 


Most readers won't accept this assumption as 
principle worth applying generally. It would 
produce a chaotic or arbitrary world, like that 
of the Queen of Hearts in Alice’s Adventures 
in Wonderland (“Off with the heads of anyone 
I don’t like!”). So far, so good. But how does 
understanding warrants make you better at 
writing arguments? The answer is simple: 
warrants tell you what arguments you have 

to make and at what level you have to make 
them. If your warrant isn’t controversial, you 
can immediately begin to defend your claim. 


THE ENGLISH LITERACY REQUIRED OF FIRST YEAR COMMUNITY COLLEGE STUDENTS 


49 


B Text Complexity Report 


But if your warrant is controversial, you 
must first defend the warrant—or modify it 
or look for better assumptions on which to 
support the claim. Building an argument on 
a weak warrant is like building a house on a 
questionable foundation. Sooner or later, the 
structure will crack. (155) 


General Findings 


Correlation of Metrics Chall et al’s findings regarding 
the relative comparability of complexity ratings 
across the different metrics held true. In other 

words, there was wide agreement between all the 
metrics about which texts were most difficult and 
which posed challenges that weren’t as significant to 
readers. In short, our findings (summarized below in 


Table One) concurred with Chall’s point that 


“different methods of estimating difficulty 
tend to produce similar results. That is, 

tested reading comprehension, judgments of 
teachers, judgments of readers, and scores from 
readability formulas on the same materials 
produced essentially the same estimates of the 
difficulty of the texts” (ix). 


And with few exceptions, scoring using the QATD 
also largely mapped to ratings on the computer-based 
tools. Given its prevalent use in schools, it seems 
worthwhile to determine whether Lexile scoring of 
these texts would map to our findings. Among the 
metrics, the Gunning Fog skewed consistently high 
with scores two to three levels above the others. 

This is not an issue though, given that Gunning 

Fog scores largely correlated with the other metrics, 
e.g. texts deemed as difficult on the Gunning Fog 
were by and large deemed difficult on the other 
metrics. Gunning Fog uses a higher standard for 
comprehension than the other formulas so the higher 
scores are typical for this tool. 
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Comparable Levels of Text Complexity The difficulty 
levels and kinds of challenges posed by textbooks 
for academic and technical disciplines used in 
community colleges and open admissions universities 
are comparable across disciplines. This means that 
the amount and kind of technical vocabulary, the 
sentence and text structures, levels of cohesion, 
and the importance of background knowledge for 
comprehension are similar. This seems a critical 
point, one that is generally not acknowledged or 
appreciated. A few scored lower (e.g, Composition 
and Early Childhood), a few were higher (e.g., 
Criminal Justice and Nursing), but on the whole 
reading in all the disciplines present similar and 
comparable challenges to students. 


Disciplinary Demands On the whole, all the 
textbooks we reviewed pose serious challenges for 
students, especially in college classrooms where they will 
be largely unassisted. Perhaps the greatest among these 
is cultivating the capacity to retain, account for, 

and apply large amounts of declarative knowledge 
that very often appear to have distant and unclear 
connections to real contexts. Disciplines featuring 
less difficult texts (e.g. Early Childhood and 
Composition) by and large lacked the dense passages 
featuring large amounts of discipline-specific 
vocabulary that marked the more difficult texts 

(e.g. Accounting, Criminal Justice, and Automotive 
Mechanics). Textbooks in math, computer science, 
and automotive mechanics featured large numbers 
of often very complicated “documents”—tables, 
graphs, maps, and lists that were used to amplify, 
summarize, or illustrate information provided in 
prose. These documents impact the complexity of a 
text. It is important to note that none of the standard 
complexity metrics have the capacity to account 

for document complexity. This gap obscures, for 
instance, the fact that automotive technology texts, 
which all the metrics marked as difficult but less so 
than other disciplines, might in fact pose significant 
challenges for readers unrecognized by traditional 
readability metrics. (During our work on this report 
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we conducted a cursory exploration of the PMOSE/ 
IKIRSCH tool, a metric that allows a reader to 
distinguish between different “documents” and to 


be able to read that text successfully in a high school 
classroom where they receive instructional support 
that aids comprehension is likely struggle with that 


surmise the impact they might have on readers. This 
work is summarized in Appendix II). 


text in the less-supported college environment. 


2. Students must be able to negotiate or process 
: large amounts of new information: Students 

Conclusions need capacities for building mental models of 

Text complexity at the college level—in other disciplines, for managing the conceptual loads 

words, the challenge level presented to students required of that work. The density of ideas in 

by entry-level textbooks—is distinguished by the texts in many disciplines—from Auto Mechanics 

following factors: heavy content, lots of specialized to Nursing to Criminal Justice—suggests 

vocabulary, sophisticated text structures (including that students must be equipped with both the 
the relationships between parts and subparts in a stamina and skills to absorb and retain large 
text), and long and relatively complicated sentences. amounts of declarative knowledge and to be 

All of the textbooks analyzed in this study are marked able—presumably—to apply that knowledge in 

to some degree by these features. Students who will be academic, clinical or laboratory settings. High 

successful readers of these textbooks, even the ones school teachers should imagine that one of their 
identified as least difficult, must have the following responsibilities is to help students learn to process 
capacities: large amounts of new material independently. 

1. Students must be able to read difficult texts in 3. Students must have sophisticated academic 
unsupported environments: The search for a vocabularies: A student’s ability to negotiate texts 
definitive response to the question “is this text with large proportions of difficult words is often 
high school level or college level according to regarded as the best predictor of comprehension. 
this metric” must be informed by the Students need to be able to negotiate difficult, 
following realities: often technical vocabulary. It is important 

to note that these are not word recognition 
e ‘The biggest difference between high challenges in which a reader is faced with terms 
school and College reading is that in high they would likely encounter in everyday life, but 
school a students’ reading of text is largely word identification challenges which often entail 
supported through instructional activities familiarity with concepts and knowledge contexts 
and regular interactions with a teacher. that would be quite disorienting to someone who 
lacks appropriate prior knowledge or experience. 
e ‘The reading volume in college is 
significantly greater in college. 4. Students must read a lot and in multiple 


College students are expected to read 
significantly greater amounts of material 
without support. 


‘The implications of these points are significant: if 
a text used in a community college classroom is 
written at a “high school level,” a student who might 


genres: Reading volume and breadth is very 
important. Familiarity with text structures 
and size of reading vocabulary directly 
corresponds to success on reading achievement 
tests. Regular reading experience also leads 

to the development of a kind of stamina that 
would likely be important a student’s ability 
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to negotiate long and complicated texts. 
Self-selected reading is important, though 
the unique demands posed by nonnarrative 
texts suggests that schools should pay careful 
attention to providing students with regular 
and deep access to informational materials in 
a wide range of content areas. 


5. Students must have experience working with 
difficult academic texts in classroom settings: 
‘The implications for teaching and learning are 
not to be minimized. By and large, a student’s 
ability to manage and make sense of complex 
texts is directly impacted by their experience 
of those texts in classroom settings. Teachers 
of all content areas should take care to make 
the structures of knowledge in their disciplines 
explicit to students. The importance of 
professional development for teachers focused 
on helping them teach students to read texts 
in their disciplines cannot be underestimated. 
One question to consider is whether a tool 
like Cohmetrix—which offers a much more 
nuanced picture of the challenges posed by 
a given text—could be used to help teachers 
spot and appreciate difficulty in texts in ways 
that would help them more precisely prepare 
students for difficult reading by equipping them 
with information and insights that allow them 
to coach students through specific kinds of 
difficulty. A student’s experience with a text is 
also directly and powerfully shaped by the task 
or purpose for which the student is reading. 


6. Students must know how to recognize and use text 
structures to scaffold their comprehension: This 
ability will directly impact a student’s ability 
to comprehend complex texts in content-area 
classes. The extent to which students can see 
and understand connections between parts 
and subparts, paragraphs, and sentences within 
paragraphs directly affects their abilities to 
organize that information in ways that allow 
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them to access or apply it in both academic and 
real-world contexts. Students would benefit 
from explicit instruction about text structure, 
including the ways in which knowledge in 
different disciplines is organized and how those 
different organizational structures inform the 
way a reader moves through a text. 


Students must be able to read and comprehend 
“documents”: Document literacy is very 
important. Reading texts in college is not 
exclusively a question of students’ ability to 
read prose. Students must be able to read 

and understand the tables, charts, maps, and 
lists that supplement prose if they are to be 
successful, especially in courses or disciplines 
that require students to internalize large 
amounts of factual information. It is important 
to remember that the complexity of documents 
increases difficulty in ways that are currently not 
captured by most readability metrics. Our brief 
foray into work with the PMOSE/IKIRSCH 
tool suggests that the challenges posed by 
documents in different disciplines is not the 
same—textbooks in some disciplines use more 
and more complex versions of these non-prose 
texts than others. We do not report on the 
findings of this exercise as it was the first time 
we used the tool and our purpose was to see how 
it worked and whether it would be a useful tool 
for this project. We do think it was information 
and would recommend doing a broader scan 

of the texts through this lens. This work is 
summarized in Appendix II of this report. 
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TABLE 1 Text Difficulty Comparison: Grade Ratings for Texts in Introductory Courses in Select 
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APPENDIX 1 Individual Text Tables 


Automotive Technology Business/Accounting Criminal Justice Early Childhood Education 
20 20 17.2 a 
15 15 14.1—14.5 15 14.2 
11.5 11.5 11.8 as 
10 10 10 
5 5 5 
QATD NDC FK_ GF QATD NDC FK GF QATD NDC FK _ GF QATD NDC FK_ GF 
English Composition - English Composition - 
Anthologies Texts IT Mathematics 
20 20 20 20 
15 T3.0 15 15 13.5 15 13.9 
10.4 eae 10.7 10.1 
10" |e 10 10 10 
L ; 
QATD NDC FK_ GF 2 QATD NDC FK_ GF QATD NDC FK_ GF QATD NDC FK_ GF 
Nursing Biotech/Electrical Technology 
20 20 |e] 
15 15 


10 


5 


14 1 
126 Ginn 
10 
5 
0 


4.7 
QATD NDC FK GF 


QATD NDC 


APPENDIX 2 Documents and Text Difficulty 


During another meeting, we used the PMOSE/ 
IKIRSCH Document Readability formula to score 
graphics selected from a small sampling of textbooks 
(one to two texts per subject area, not including 
mathematics) from each discipline. 


The PMOSE/IKIRSCH Document Readability 
Formula offers a way to measure and compare print 
material mot written in prose format. The rationale 
for using the PMOSE/IKIRSCH tool was that all 
metrics of text difficulty are defined (and limited) 


54 WHAT DOES IT REALLY MEAN TO BE COLLEGE AND WORK READY? 


by the constructs they define and measure. Most 
readability formulas ignore one of the most pervasive 
characteristics of textbooks: the tables, charts, 
graphs, maps, and lists that Mosenthal and Kirsch 
call “documents.” According to Mosenthal and 
Kirsch, documents are “print materials structured as 
lists, charts, or graphic displays.” These graphics pose 
additional challenges for readers, challenges that are 
the result of density and structure. The PMOSE/ 
IKIRSCH tool rates a text via three different criteria: 


1. Structure: What is the design of the document? 
Different structures or displays (list, table, pie 
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FIGURE ONE Document from Auto Fundamentals, by Stockel, Stockel, and Johanson 
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Figure 22-29. An exploded view of a semi-floating, hypoid type, rear axle assembly. This unit has an integral ring and pinion carrie: 
(differential) housing. A—Exploded view of the differential assembly. B—Exploded view of the optional disc brake assembly. 1— 
Differential drive pinion gear nut. 2—Differential drive pinion gear washer. 3—Differential drive pinion gear yoke. 5—Differential drive 
pinion gear dirt deflector. 6—Differential drive pinion gear seal.7—Differential drive pinion gear outer bearing assembly. 9—Rear ax= 
housing drain plug. 15—Differential drive pinion gear spacer. 16—Rear axle housing assembly. 17—Rear axle vent. 18—Differentia 
drive pinion bearing assembly. 19—Shim differential drive pinion gear. 20—Gear differential ring, 22—Ditferential carrier bearing cac 
23—Ditferential drive pinion mounting gear. 24—Differential bearing shim. 25—Differential bearing spacer. 26—Differential bearing 
cap bolt/screw. 28—Differential ring gear bolt/screw. 29—Differential pinion gear shaft boli/screw, 32—Differential bearing assembly 
33-—Differential case. 34—Rear axle housing chip collecting magnet. 35—Differential pinion gear shaft. 36—Differential pinion gear 
37—Differential pinion gear thrust washer. 38—Differential side gear thrust washer. 339—Rear wheel speed sensor reluctor wheel. 
40—Differential side gear. 41—Rear axle housing bolt/screw. 43—Rear axle housing cover. 44—Rear axle shaft bearing assembly. 
45—Rear axle housing cover gasket. 46—Rear brake backing plate. 48—Rear axle housing bolt/screw. 49—Rear axle shaft bearinc 
seal. 54—Rear axle shaft lock. 58—Rear brake drum. 59—Rear brake assembly. 61—Rear wheel bolt/screw. 63—Rear axle (drum 
brake assemblies) shaft. 64—Rear brake backing plate nut. 65—Rear brake caliper assembly plate, 66—Caliper assembly mounting 
balt/screw. 67—Axile tube flange shim. 68—Rear brake caliper mounting plate nut. 69—Rear brake caliper assembly. 70—Rear axle 
(disc brake assemblies) shaft. 71—Rear brake rotor. 72—Rear brake backing plate bolt/screw. 73—Sensor plug mounting bolt/screw 
75—Wheel speed sensor assembly. (Pontiac) 
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chart, maps) are given different scores according § The PMOSE/IKIRSCH tool is designed to help a 


to overall difficulty and sophistication: are they reader determine the complexity of a given document 
simple lists? Combined lists? Intersecting lists? by using a simple point system for each of the 
Nested lists? criteria above. Figure One and Figure Two below are 
examples of documents that can be scored using the 
2. Density: How many titles and items are PMOSE/IKIRSCH Document Readability Formula. 
presented to the reader in the document? Both are examples of print materials that are not in 


sentence and paragraph format. 
3. Dependency: Does the reader need to look 
outside the document for important information 
required to make sense of the document? 


FIGURE TWO Document from Introduction to Early Childhood Education, by Marian Marion 


FIGURE 10.5 Guiding Principles of Early Childhood Curriculum Development 


Principle 1: Plan a curriculum reflecting all developmental domains 


Principle 2: Develop a curriculum with intellectual integrity and regarded as appealing and 
important to children 


Principle 3: Plan a comprehensive and integrated curriculum focusing on recognized content areas 
Principle 4: Plan a curriculum encouraging children’s social interaction 
Principle 5: Plan a curriculum with appropriately challenging and novel learning activities 


Principle 6: Plan a curriculum to meet the needs of all children 


Figure One is an example of a map that contains a ‘This also increases the complexity of the document. 
large number of nested labels. It is not uncommon Our readers determined that this document’s 

to see many documents like this in an automotive complexity level was “high” according to the 
mechanics textbook—or for that matter in a biology PMOSE/IKIRSCH scale—a scale that ranged from 
or anatomy textbook. According to the PMOSE/ “Very Low” to “Very High.” Figure Two, on the 
IKIRSCH tool, one of the things that makes this other hand, received a score of “Very Low.” It has a 
document particularly difficult is the shear number simple list structure and features only a few items 
of labels and items in the map. In addition, the and labels and does not require discipline-specific 
map requires that the reader possess a deep subject- content knowledge for comprehension. 


specific knowledge-base to make sense of the map. 
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C Community College 101 Courses: Student Assesment Instruments and 
Test/Exam Formats Used 


‘The evaluation instruments that are listed here 

are drawn from each course’s syllabus, with each 
row representing a different course at a different 
community college. The weight each instrument 
is assigned in the determination of a student's final 
grade is captured in the percentage that appears 
adjacent to each one. 


ACCOUNTING 


Student Assessment Instruments Test/Exam Formats Used 


Average of homeword marks Tests/exams are multiple choice and constructed 
Average of 4 multi-chapter tests, midterm exam and response 

final exam* 

Quizzes (15%) Exams are multiple choice and constructed response 


Homework (15%) 
2 Exams (30%) 
Final (40%) 


4 Multi-chapter exams Exams are multiple choice and constructed response 
Final exam 
Challenge problems* 


Final exam and multi-chapter tests (48%) Tests/exams are multiple choice 
Single chapter quizzes (18%) 
Textbook materials (16%) 
Group project (12%) 

Class participation (6%) 


Exams 1-3 (60%) Tests are multiple choice and constructed response 
Final exam (25%) 
Class participation (10%) 
Assignment (5%) 


Group exercises, quiz and project (30%) Final exam and quiz are multiple choice 
Homework (25%) 
3 Chapter exams (10% each) 


Quizzes, homework, projects Final exam is multiple choice and constructed 
Midterm exam response 
Final exam* 


*Weighting of instruments unknown 
**As presented in the syllabus, does not add to 100%. 


58 WHAT DOES IT REALLY MEAN TO BE COLLEGE AND WORK READY? 


Community College 101 Courses: Student Assesment Instruments C 


and Test/Exam Formats Used 


AUTO TECHNOLOGY 


Student Assessment Instruments Test/Exam Formats Used 


Homework (10%) Quizzes and exercises are constructed response or 
WorkKeys Assignment (5%) short activities 

Communication exercise (5%) Final written exam is constructed response and 
Lab exercises (30%) multiple choice 


Quizzes (10%) 

Midterm exam (10%) 
Written final exam (10%) 
Hands-on final exam (20%) 
Portfolio (5%)* 


Tests (50%) Tests/exams are multiple choice 
Daily grade which includes homework and in-class 
work (25%) 

Final Exam (25%) 


Quizzes and exams (40%) Final exam is true/false, multiple choice and essays 
Homework (10%) 

Labs (40%) 

Research paper (10%) 

Class participation (+/- 10%)** 


BIOTECH/ELECTRICAL TECHNOLOGY 


Student Assessment Instruments Test/Exam Formats Used 


Exams (40%) Exams are multiple choice, constructed response, 
Lab reports and practicals (30%) and essay 

Homework and quizzes (15%) 
Student project (10%) 

Notes and notebooks (5%) 


Tests/exams* Test/exams are multiple choice and fill in the blank 
Other instruments - unknown 


BUSINESS 


Student Assessment Instruments Test/Exam Formats Used 


Midterm exam (30%) Final exam is multiple choice and essay 
Final exam (40%) 

Oral presentations (15%) 

ePortfolio (5%) 

Overall (participation, homework) (10%) 


*Weighting of instruments unknown 
**As presented in the syllabus, does not add to 100%. 
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Test/Exam Formats Used 


Student Assessment Instruments Test/Exam Formats Used 


Quizzes and exams (50%) Final exam is multiple choice 
3 Projects (30%) 
Homework (20%) 


Quizzes (25%) Quizzes and tests are multiple choice 
Team project (20%) 
5 Tests (40%) 

Participation (15%) 


Tests (30%) Format of tests/exams unknown 
Final exam (40%) 
Project (30%) 


COMPUTER PROGRAMMING 


Student Assessment Instruments Test/Exam Formats Used 
Final exam (20%) Midterm exam is a paper 
Midterm midterm (20%) Final exam is a programming assignment 
Quizzes (20%) 


Homework (15%) 
ePortfolio (15%) 
Participation (10%) 


Quizzes (20%) Midterm exam and final exam are constructed response 
Assignments (30%) 
Participation/homework (10%) 
Midterm exam (15%) 

Final exam (25%) 


CRIMINAL JUSTICE 


Student Assessment Instruments Test/Exam Formats Used 


3 tests (75%) Tests/exams are multiple choice. 
Final exam (25%) 


Assignments (30%) Exams are true/false and multiple choice 
Class participation/papers (20%) 
Midterm exam (20%) 

Final exam (20%) 

WorkKeys pretest (10%) 
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Community College 101 Courses: Student Assesment Instruments C 


and Test/Exam Formats Used 


Student Assessment Instruments Test/Exam Formats Used 


Midterm (25%) Final exam is multiple choice 
Final exam (25%) 
Term paper (15%) 
Syllabus test (4%) 
Presentation (4%) 
Quizzes (8%) 
Participation (10%)** 


2 Exams (20%) Final exam is multiple choice, short answer, and essay 
Final exam (30%) 
3 Papers (30%)** 


Midterm exam (25%) Final exam is short answer and essay 
Final exam (25%) 

2 Assignments (40%) 
Participation (10%) 


Chapter tests (40%) Final exam is multiple choice and essay 
Final exam (20%) 

In-class assignments (20%) 
Quizzes/other (20%) 


Tests (40%) Tests are multiple choice and short essays 
Discussion assignments (30%) 
Group project (20%) 

Attendance/participation (10%) 


EARLY CHILDHOOD EDUCATION 


Student Assessment Instruments Test/Exam Formats Used 


Attendance and class participation (24%) Format of midterm and final review unknown 
Observations of early childhood classrooms (20%) 
Self-assessment (6%) 

Class presentations (15%) 

Journal reflections (15%) 

Midterm review (10%) 

Final review (10%) 


Fieldwork Tasks (25%) Format of quizzes unknown 
Quizzes (20%) 

Orientation training for Childhood Development 
Associate (CDA) certificate (7.5%) 

CDA Portfolio (autobiography, competency statement 
and collection of resources) (30%) 

Personal philosophy of education (2.5%) 

Class participation (10%) 


**As presented in the syllabus, does not add to 100%. 
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C Community College 101 Courses: Student Assesment Instruments and 
Test/Exam Formats Used 


Student Assessment Instruments Test/Exam Formats Used 


Term paper (20%) Exams are essay exams 
3 Exams (30%) 

In-class writing assignments (10%) 
Final exam (25%) 

Class participation (15%) 

Extra credit writing assignments (+10%) 


3 Tests (60%) Tests include multiple-choice, true/false 
Observation project (20%) and short answer 
In-class papers (20%) 


3 Exams (40%) Exams and quizzes are multiple choice 
Quizzes (20%) 

Term research project (13%) 
Final exam (27%) 


Exams Exams are multiple choice 
Other instruments unknown* 


3 Tests (60%) Tests are multiple choice, true/false and matching 
Journal critique (10%) 

Oral summary of newspaper article (5%) 
Written classroom observations (20%) 

2 One-page reaction papers (5%) 

Extra credit book chapter notes (3%) 


INFORMATION TECHNOLOGY 


Student Assessment Instruments Test/Exam Formats Used 


14 Chapter quizzes (25%) Exams are multiple choice 
Lab assignments (40%) 
Midterm exam (10%) 
Final exam (20%) 
Honor’s project (5%) 


Final exam (20%) Exams are constructed response 
Midterm exam (20%) 

Quizzes (20%) 

Homework (15%) 

ePortfolio (15%) 

Participation in Blackboard discussions (10%) 


4O Class and lab assignments (67%) Exams are multiple choice 
3 Exams (33%) 
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Community College 101 Courses: Student Assesment Instruments C 


and Test/Exam Formats Used 


Student Assessment Instruments Test/Exam Formats Used 


4 Exams (40%) Exams are multiple choice 
10 Quizzes (30%) 

20 Assignments (30%) 

3 Blackboard discussions (5% bonus) 


Assignments (25%) Exams are short answer, fill in the blank 
Concept exams (20%) or multiple choice 

3 Microsoft exams (10%) 
Final exam (15%) 


NURSING 


Student Assessment Instruments Test/Exam Formats Used 


Unit quizzes/exams (40%) Exams and quizzes are multiple choice 
Comprehensive Theory and Clinical final exam (25%) 
Exit exam (5%) 

Oral presentation (5%) 

Written assignments (5%) 

Outside assignments (10%) 

Lab Practicum (10%) 


Exams/tests Exams/tests are multiple choice 
Other instruments - unknown* 


Chapter exams (76%) Exams are multiple choice, true/false or matching 
Final exam (16%) with occasional essay questions 
Quizzes/projects (8%) 


6 Exams, 2 lab practical exams, and final exam Exams are multiple choice 
(70%) 
Medical terminology quizzes and homework (30%) 


3 Unit exams (20% each) Exams are multiple choice 
Final exam (40%) 


Tests (75%) Tests/exams are multiple choice 
Final exam (25%) 
Lab is pass/fail 


*Weighting of instruments unknown 
**As presented in the syllabus, does not add to 100%. 
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Business Plan for an Startup Business Page 1 of 26 
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Example 3 - Business Plan Excerpt D 


NB: ONLY PARTS I-IV ARE INCLUDED IN THIS EXCERPT 


Business Plan for an Startup Business Page 2 of 26 
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II Executive Summary 
| 


Owners: . 
* 0 
Product: 
“ We operate an On-line service to handle Firearms and accessories for the 
Ce EINE EERSSIIISRINIIIeeen ee 
Sportsman and Collector. We will assist in the legal transfer as a i 
SS ee 
Federal Firearm License holder for those purchasing firearms. 
Customers: 


“* Our customers will be Sportsmen and Collectors who appreciate fine 


firearms. 
Growth in the Industry: 


¢ The industry will continue to grow as more women—51 percent of the adult 
populations as of the 2000 census—take advantage of the services the 
industry offers. As the number of women living alone continues to increase 
so has their demand for the products and training provided by the industry. 


“ New government regulations applying to the transfer of firearms between 
individuals will serve to grow the number of FFL holder needed. As more Ged ; 
states adopt Canceled Carry Permits the demand for the services and 
products provided by the industry will also continue to grow. 


¢ The explosion of Internet use in America increases the number of possible 
clients daily. The Internet has proved invaluable in showcasing the products 
and services offered by the industry and will continue to do so. 


Funding: 


We are able to self fund the Startup of this business. ee 
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Business Plan for an Startup Business Page 4 of 26 


iil General Company Description 
General Company Description: 


“ Wewill handle the legal responsibilities for the transfer of sporting arms 
from the seller to the buyer. We will order and locate firearms for the 
sportsman or collector. We will sell firearms we acquired or accepted on 


consignment. We will provide estimates of value for insurance or sale 
purposes. Ge 


Mission Statement: 


© Toprovide our customers with a trusted and reliable avenue for the 
acquisition and transfer of sporting arms. To accurately place a value on 
collections for insurance and estates purposes. To reinforce the positive and 
family friendly values of the sport. To educate users in the safe handling and 


use of firearms. f, a) 
ane 
Company Goals and Objectives: 


% Goals: 


|. To grow an on-line business supplying firearms to the sportsman and 


Pea 
collector. 


ll. To grow this business—by expanding the product line—to a level capable 
of generating profits in excess of $100,000.00 annually  -—~ 


Ill. To grow this business to a level that will justify expansion to-a brick and 


mortar operation. 
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< Objectives 
1. Prepare for startup: 
o Establish an LLC ~~ 


© Obtain FFL—application process can run 6 months or longer--and 
— 
any necessary permits needed to conduct this type of business. 


o Obtain Gun Safe and digital storage equipment. ee 
400) 
o Move fram DSL to a Cable internet connection. ya 


© Obtain necessary equipment and setup accounts with a provider 


of Master Card and Visa services. 


ll. Make final adjustments to Business Plan in preparation—to coincide with 
or follow shortly after my Graduation from CNM and finding employment 
and Linda’s retirement—for a launch date of April, 2011. — 


Establish a presence on-line—using the services of Guns America and 


Guns International--listing new products. We sre \ ' 
IV. Within six months of startup establish a Web ordering site separate from 


Guns America and Guns International. Direct customers to this site within 
the listing posted on Guns America and Guns international. 


V. Within twelve months of startup begin listing—used and collector 
i 


firearms—items related to the secondary market. 


Vi. 


Within twenty-four months begin listing new products that have been 
purchased new and enhanced by—Gunsmith—John Farner of Eagle 
Machine-Gunsmithing & Machining Services. a 


68 WHAT DOES IT REALLY MEAN TO BE COLLEGE AND WORK READY? 


Example 3 - Business Plan Excerpt D 


Business Plan for an Startup Business Page 6 of 26 


Business Philosophy: 


* To provide quality service to the customer; establishing a working i 
relationship that is enjoyable to both the customer and me. i 


Our products and services will be marketed to firearm enthusiasts, 
collectors, or their estates. 


Industry Description: 


The procurement and collection of firearms has changed drastically since the gun 
control act of 1968 was adopted. in the not too distant future ! believe the government 
will impose restrictions on the transfer of firearms between individuals. These 
restrictions will force transfers to be handled by a Federal Firearm License holder. 
Collectors are aging and the changing economy Is and will continue to force many to sell 
their collections. A growing segment of the society, women, have begun to seek 
training in the use of firearms. By obtaining a FFL we are able to satisfy the legal duties 
involved in the procurement and transfer of firearms. As a NRA certified instructor we 
will be able to participate in educated recommendations to new enthusiasts. 


Y 
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{ 
IV Products and Services ar. 

We will act as a dealer for several of the large firearm distributers. In this 
capacity, we will receive the firearm from the distributer and handle the legal transfer to 
the individual. We will list on-line any firearms we acquire or accept on consignment. 
When a firearm is purchased on-line we will handle the legal transfer to a FFL holder in 
the buyer's home state. We will provide written estimates of collections or individual 
items. All estimates will be based on wholesale value. We will offer free of charge 


recommendations for firearm selection and recommended training facilities. 
Strengths: 


* By operating as an On-line service the costs associated with a brick and 
mortar establishment are avoided. on 


“4 We possess a comprehensive knowledge of the products to be provided. 
Because we are a Home Bases operation there is very little overhead. 


¢ Acting as a Dealer for the larger Wholesalers eliminates the burden of new 
product inventory. 


“ Operating as an On-line service allows the customer to place orders 24 


hours a day 7 days a week. 


“ By not holding new product inventory funds can be directed to the 
accusation of secondary—used and collector firearms—market inventory. 
This segment of the market offers an opportunity for a greater profit 
margin. 
Disadvantages: 


Lack of a brick and mortar facility. ce 
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Pricing: 


There is no licensing fee to act as a desler for any of the large firearms distributors. Our fee for 
a transfer will be $35.00. The price of ordered firearms will be cost plus 30 percent. The price 
on consignment firearms will be wholesale value plus 30 percent. The cost of an appraisal will 
be on an individual basics starting at $10. A locator fee of 30 percent of the retail value, or 
$50.00 per research hour, will be applied for the location of collector firearms— 


V§Market Research 


In 2000 the United States Census Bureau projected a population of 300,000,000 
to be reached in 2006. Of that 300 million, 200 million are of Adult age and potentially 
able to purchase a firearm. If 50 percent of that 200 million are excluded by choice, 
criminal activity, or due to mental illness the number of possible firearm owners falls to 
100 million. This number is born-out by research preformed by the Gallup Poll. In fan 
September, 2000, the Gallup Poll placed the number of gun owners in America at 


wast 


between 77 million and 90 million. Research preformed by the Pew Research Center and 
published in June, 2008, placed the number of adult Americans with Internet access at 
110 million. Therefore, by marketing our services on the Internet we will directly reach 
approximately 55 million of the 100 million gun owners in America today. With the 
continuing growth of the Internet the number of potential clients will continue to grow. 


Economics: 


The United States Census Bureau projected a population of 300,000,000 to be reached in 2006. 
Of that population 1/3 would be 21 years old and younger. Currently 50% of the adult 
population own firearms. Based on these numbers there is a potential market between ( 
50,000,000 and 100,000,000 buyers for this product. (ne0 
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Current Demand in Target Market: 


Currently there is a very high demand in the market for this product. The fear of government 
regulation has created an artificial shortage, brought about by the hording of firearms and 


ammunition. 
Trends in Target Market: 


Demand for this product, particularly those with a military appearance, will remain high for the 
foreseeable future. | expect it to fall to reasonable levels, upon the exit of the current 
administration, in 2012. _ 


Growth potential: 
The potential for growth in the business is enormous. 


The changing Federal regulations of this industry have driven many brick and 
mortar shops out of business. This situation has been amplified by the aging of current 
shop owners. Many current shops where founded by World War tl and Korean War 
veterans. While many of their sons chose to continue the business; their grandchildren 


have chosen more glamorous and more politically correct career paths. ¢ ¢ 
ihe atch racing castrate! sau ae tages Rare nares cea 


Many former and current owners failed and continue to fail in adapting to newer 
business models. Use of the Internet and modern shipping methods could have saved 
many older shops. It is no longer necessary to hold large inventors on-site. Use of the 
Internet and modern shipping methods allow the client to view the product and delivery 
can accrue in as little as three day. Inventory that is on site can be displayed to millions 


yey 4087 


by way of the Internet. 
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Essay Prompt... cc cccccccccccccceneneeeeennnees 74 
Essay Ruri sic veceivestecaciasded nee ces 74-75 
Student Reponse...ccccccccccvcvcvcvcvcvcuens 76-79 


CJ101 FINAL EXAM, ESSAY QUESTION 
Give the pros and cons to the death penalty; after that give your opinion. 


Your paper will use 12 point font, the same size as these letters. WRITE AN APA STYLE 
PAPER, cover sheet, APA page numbering & USE A SEPARATE SHEET TO LIST YOUR 
REFERENCES. For example: 


References 
American Psychological Association (2010). Free tutorial on the basic APA style. Retrieved 
April 19, 2010, from http://www. e.org/apa-style-help. 


Inciardi, J. A. (2010). Criminal justice (9" ed.). New York: McGraw-Hill 

Show direct quotes such as, Inciardi (2010) said, “On January 20, 1864, William Barnet 
and Sandy Kavanagh were executed in the Vermont State Prison for the crime of murder” (p. 
383). Notice there was only one period and that was after the page number. For longer quotes 
such as, Inciardi (2010) said: 


On January 20, 1864, William Barnet and Sandy Kavanagh were executed in Vermont 
State Prison for the crime of murder. During the next 100 or so years, through 1967, a 
total of 5,707 state-imposed death sentences were carried out across the country. (p. 383) 


Notice on this indentation that the page number is after the quote. The period comes at the end 
of the sentences and since the quote is indented, quotation marks are not used. 


74 WHAT DOES IT REA 


I will be looking at: 


(1) Content; is it even, balanced information (THE CONTENT SHOULD BE AT LEAST 1 
PAGE) 


(2) Organization; the ideas are arranged logically 


(3) Purpose; the writer’s purpose is readily apparent 


(4) Use of references; compelling evidence is given (YOU MAY USE YOUR TEXTBOOK, 
PROFESSIONAL JOURNALS, EDUCATIONAL MOVIES). HOWEVER, ONLY ONE 
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REFERENCE IS REQUIRED. 
(5) Voice; writing is compelling 
(6) Tone; the tone is consistently professional 


(7) Sentence structure; are well phrased and varied in length (NO EXCEPTIONALLY LONG 
SENTENCES). 


(8) Word choice; is consistently precise (SAME TENSE IN A SENTENCE: PAST, 
PRESENT, OR FUTURE. ALL WORDS IN THE SAME SENTENCE ARE SINGULAR 
OR PLURAL) 


(9) Grammar, spelling and mechanics; are free of errors (AVOID USING “YOU” AND 
USING “BUT” IN THE MIDDLE OF A SENTENCE. THE WORD “HOWEVER” IS A 
BETTER WORD TO USE). 


(10) Critical thinking; accurately interprets evidence 

(11) Your paper will be titled The Pros and Cons of the Death Penalty 

(12) Your page numbering will be Death Penalty 1, Death Penalty 2, Death Penalty 3, etc. 
(13) Direct quotes ARE NOT REQUIRED. 

(14) Use Spell Check and Grammar Check 

(15) Have someone else who is a good writer to READ YOUR PAPER. 


(16) THIS ESSAY/PAPER IS WORTH 20% OF YOUR FINAL EXAM 


(17) No more than 20% of your paper should be direct quotes. However, direct quotations are 
not required. 
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The Pros and Cons of the Death Penalty 


By 


CJ, 101, Introduction to Criminal Justice 
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Death Penalty 2 
The Pros and Cons of the Death Penalty 


Jae Inciardi (2007) said, “The death Penalty Debate, historically the arguments for or 
against capital punishment have revolved around the issues of economics, retribution, public 
opinion, community protection, deterrence, irreversibility, discrimination, protection of the 
criminal justice system, brutalization, and cruel and unusual punishment“ (p. 440). There are 
different ways to execute someone who has been given the death penalty. 

From many decisions from 1878 through 1953, the Supreme Court upheld different types of 
executions such as; hanging, shooting, electrocution, and the use of lethal gas. 

There are many different Pros and Cons for capital punishment. First, the Pros are usually 
the victims of the crime, and the cons are usually the convicted and family members. The first 
execution by electrocution took place at Auburn Prison, Auburn, New York on August 6, 1890. 
The prisoner, William Kemmler, was executed for murder. 

The cases that follow stated in Inciardi (2007): 

Lockhart v. McCree, decided in 1986, the court asserted that even if juries that support 

the death penalty are “conviction prone,” this in itself does not violate any constitutional 

provisions (p. 446). McCleskey v. Kemp, decided in 1987, the Court held that statistical 

evidence of racial discrimination in death sentencing could not, in and of itself, establish a 

violation of the Eighth and/or Fourteenth Amendments. The court further held that to 

obtain relief, a defendant must prove either (1) that decision makers in his or her case 
acted with discriminatory purpose or (2) that the legislature enacted or maintained the 

death penalty statute because of an anticipated racially discriminatory effect (p. 446), 

The cons for the argument is as follow; Inciardi (2007) said,” In the fall of 1971, Furman 


v. Georgia, Jackson v. Georgia, and Branch v. Texas were brought before the court on the 


ground that the death sentences ordered were “cruel and unusual” because of the arbitrary and 
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Death Penalty 3 
discriminatory manner in which such sanctions had been imposed in the past for the crimes of 
murder and rape” (p. 433). 

For every pro, there is a con against the death penalty. For me, it will always be Pro, if 
someone can stand and take a life, then he or she should be ready to lose their own. If they have 
so much contempt and disrespect for life then they will commit murder again, and next time it 


could be a family member, a close friend, or even yourself. 
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Death Penalty 4 
References 


Inciardi, J. A. (2007) Criminal Justice. New York: McGraw-Hill 
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Argument Essay 


Imagine that a congressman has introduced this new bill to Congress: 
“The United States government shall guarantee all American citizens a minimum 
income of $10,000 a year for a single person, $12,000 a year for a married couple, 
and $15,000-20,000 a year for a family with children (depending on the number 
of children). Every adult citizen who makes more than the minimum income shall 
pay taxes on the surplus. Every citizen who makes less than the minimum income 
shall receive a refund from the government to bring their income up to the 
minimum.” 


In your essay, argue that the government should, or should not, pass this bill. 


In the columns below, brainstorm supporting arguments for your thesis, be 
arguments, and your rebuttals to the opposing argument. 


Thesis: 
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ENG 101/107 ESSAY COVER SHEET 


Name 


Assignment Due Date 


Essay Grade 


To receive a grade, the topic and mode must be appropriate for the assignment, and 


the essay must be typed in Standard Form. 
Possible Your 


Content (100 points possible): Points Points 
1. There is a short, clear title. 5 points 
2. The first paragraph attracts the reader’s interest and introduces 
the topic well. 20 points 
3. The thesis is well written and clearly stated in the first paragraph. 20 points 
4. The body is well organized and supports the thesis well. 40 points 
5. The conclusion effectively ends the essay 15 points 
Total Points 
Point Scale for Editing 
+5 points 0 -5 points -10 points -20 points -40 points 
Qerrors 1-3 errors 4-Serrors 6-7errors 8-9errors 10 ormore 
Your Editing Score 
Point Scale for Length 
0 -10 points -20 points -30 points -40 points -50 points 
480-500 450-479 400-449 350-399 300-349 Less than 300 
YourLength Score 
Your Essay Grade 
Comments: 
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October 27, 2010 
ENG 101 


Minimum Income 


In the U.S. there are many people in need of help, but there are also people who take 
advantage of the help our government offers. Many people like to rely on government funding 
because they know they can have it easy that way. There is a new bill that states that the U.S. 
should give a yearly income of $10,000 to a single person, $12,000 for a married couple, and 
$15,000 - $20,000 for a family depending on how many children they have. Giving families a 
minimum wage will help them at least to survive and be able to bring themselves back up to a 
more settled situation. The United States government must guarantee all American citizens a 


minimum income. 


Many people would be against this bill because they say that this allows people to be 
lazy and want to just receive the money. | believe that most people don’t want to just barely 
survive in this world. Here in our country, there is a drive to want to persevere and do the most 
you can with your life and your family’s life. The majority doesn’t want to have the least, they 
want to be wealthy and this just helps those get a good start to becoming economically better. 
The minimum income would help people barely survive and there would not be as many 


homeless people and families out there. 


This bill helps our government with unnecessary spending. Having this minimum income 
means that there will be no programs people used for economic issues. There are many cases 


of people lying just to be able to receive the funding from the government, which meant they 


were giving money away to ones who didn’t need it. The minimum income forces people to use 
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October 27, 2010 
ENG 101 


that money wisely because when that’s gone there is nowhere else to face. Others might say 
that this isn’t true because they had ways to know that people actually need the help from 
these programs, but we all know that people find any way possible to get fake proof of needing 


help of the government. That is why a minimum income helps immensely. 


There are also many against the minimum income because the bill states that citizens 
who make more than the minimum income shall pay taxes on the surplus. Whether there is a 
minimum income or not, we all have to pay taxes either way. Taxes have always been there to 
help out the country in what is needed. What is being misunderstood is that this helps the 
country greatly economically since they are not going to have to pay for all the programs that 


gave free food and money. 


In the end, | believe that everybody receiving a minimum income will benefit our 
country greatly. They will not have to pay for all of these programs and we will have less 


families struggling just to survive and be able to feed their families. That is what we are here 


for, to be able to live a good life and be able to achieve the “American Dream.” 
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